NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability

NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability

Duo Xu, Faramarz Fekri

Abstract Recently Large Language Models (LLMs) have been increasingly deployed as autonomous agents in applications such as self-reflection, retrieval-augmented generation, and scientific discovery. In these settings, agents must act based on limited observations rather than full environmental states, leading to partial observability. This introduces several key challenges: belief state inference, task objective misalignment, and planning under uncertainty. Prior approaches typically condition actions on full or summarized action-observation histories whose redundant and irrelevant information can mislead the decision making of LLM agent. Inspired by human cognition, we propose a novel Ne uro- Sy mbolic F ast- S low thinking (NeSyFS) framework for LLM agent, addressing the challenges introduced by partial observability in a unified approach. We use a knowledge graph (KG) to represent the belief state, providing triplets as context for every module of NeSyFS. The fast-thinking module performs reactive action, while slow-thinking conducts a new uncertainty-aware planning by following the high-level structure of twisted sequential Monte Carlo (TSMC) algorithm. To mitigate the misalignment of task objective, a reflection module is used to reflect fast-thinking actions, and also switches to the slow-thinking module whenever reactive actions repeatedly fail. Experiments on three representative benchmarks, i.e. ALFWorld, Webshop, and ScienceWorld, demonstrate significant advantages over previous methods.

Figure 1: Comparison of previous and NeSyFS agent. Previous agent uses full or summarized interaction history as context for decision making, while our agent updates a KG-based memory dynamically and uses KG-retrieved triplets as context. The KG 𝒢 t − 1 \mathcal{G}{t-1} is updated with action a t a{t} and observation o t o{t} , i.e. 𝒢 t = Update ​ ( 𝒢 t − 1 , o t , a t ) \mathcal{G}{t}=\text{Update}(\mathcal{G}{t-1},o{t},a{t}) . The triplets related with solving task η \eta are retrieved from KG, i.e. τ t = ℛ ​ ( 𝒢 t , η ) \tau{t}=\mathcal{R}(\mathcal{G}_{t},\eta) , which can provide structured context to help LLM make correct decision.

Introduction

Large Language Models (LLMs) have achieved notable success in agentic applications by extending beyond static text generation to support autonomous decision-making, enabling capabilities such as tool use (Schick et al. 2023 ) , retrieval-augmented reasoning (Gao et al. 2023 ) , multi-step planning (Rawat et al. 2025 ) , and self-reflection (Renze and Guven 2024 ) in complex environments. These advances have allowed LLM agents to perform tasks in domains ranging from embodied interaction to software automation (Jin et al. 2024 ) and scientific workflows (Yamada et al. 2025 ) . However, a fundamental challenge arises from partial observability: unlike supervised settings with complete input context, agentic environments typically reveal only local and immediate observations, while the true underlying state remains latent (Shridhar et al. 2020 ; Yao et al. 2022a ) . As a result, LLM agents must infer a belief over the hidden state from incomplete and potentially noisy interaction histories (Xi et al. 2024 ; Ma et al. 2024 ) , which are often long and difficult to compress effectively within limited context windows. In addition, since the agent must act without access to the true underlying state, increased amount of uncertainty is introduced, and planning under uncertainty becomes a challenge for the LLM agent. Moreover, the uncertainty and missing state information from partial observability can make the LLM agent produce actions misaligned with task objective, degrading the performance of task completion.

In this work, we propose a novel neuro-symbolic fast–slow thinking framework (NeSyFS) for LLM agents that addresses the challenges of partial observability in a unified manner, enabling latent state inference, mitigating task-objective misalignment, and supporting planning under uncertainty. Unlike prior full-history-based or summarization-based approaches that rely on unstructured memory, we employ a knowledge graph (KG) to represent the latent state of the environment, thereby encoding interaction histories into a structured and symbolic memory. Specifically, each incoming observation is transformed into triplets that capture relationships and attributes of entities and events. Newly acquired facts are incorporated into the KG, while outdated triplets in KG are pruned, ensuring a consistent and dynamically updated representation of the latent state. The KG-retrieved triplets are used as context in every module of NeSyFS.

Furthermore, inspired by human cognition (Kahneman 2011 ) , the proposed framework decomposes decision-making into fast and slow thinking processes. The fast thinking module generates reactive actions by retrieving relevant triplets from the KG to construct a symbolic approximation of the latent state. In contrast, the slow thinking module performs deliberative planning under uncertainty. Slow thinking is used only when fast-thinking decision making has too much uncertainty or is unreliable. To this end, we propose a new uncertainty-aware planning algorithm which follows the high-level structure of twisted sequential Monte Carlo (TSMC) (Del Moral et al. 2006 ) . Specifically, it maintains a set of particles and uses resampling to guide them toward the target distribution modeling task-completing trajectories. LLM is prompted to predict next observation and existence of task progress in the state transition. The sampling mechanism of this method can make the particles robust to mistakes in LLM-based evaluations of task progress, and reach a good balance between exploration and exploitation. As far as we know, we are the first to develop a TSMC-style planning method for LLM agent.

Although fast thinking emphasizes locally plausible actions, it does not explicitly guarantee alignment with global task objectives. To address this limitation, we incorporate a reflection module that evaluates the consistency of fast-thinking actions with the task objective, conditioned on relevant triplets retrieved from the KG. To control computational overhead, slow thinking is invoked only when fast-thinking actions repeatedly fail in this reflective evaluation. With KG-provided triplets as context, stepwise reflection is enabled and is more efficient and robust than conventional trajectory-wise reflection.

Our contributions are summarized as follows.

• We propose a neuro-symbolic fast-slow thinking framework that addresses challenges introduced by partial observability in a unifying manner.

• Based on the context retrieved from the KG, a neuro-symbolic TSMC-style planning algorithm is introduced within the slow-thinking module to address the uncertainty under partial observability.

• We design a KG-augmented reflection module that aligns fast-thinking actions with task objectives. Using KG-retrieved triplets, the module enables stepwise reflection that is more efficient and robust than conventional trajectory-level reflection.

• Experiments on three benchmarks demonstrate that using KG-retrieved triplets as context improves the LLM performance on decision-making, reflection, and world modeling.

Preliminary Background

Partial Observability

We consider agentic tasks in which an autonomous LLM responds to a user query through iterative interaction with an external environment ℰ \mathcal{E} . Each episode begins with a user query and unfolds over a finite horizon H H .

Such agentic tasks can be naturally formulated as a Partially Observable Markov Decision Process (POMDP) (He et al. 2024 ; Zhang et al. 2025 ) , represented by the tuple ℳ = ( 𝒮 , 𝒜 , 𝒪 , 𝒯 , O , R ) \mathcal{M}=(\mathcal{S},\mathcal{A},\mathcal{O},\mathcal{T},O,R) . Here, 𝒮 \mathcal{S} denotes the latent state space of the environment, 𝒜 \mathcal{A} the action space, and 𝒪 \mathcal{O} the observation space consisting of textual environmental response. The environment dynamics are governed by the transition function 𝒯 ​ ( s ′ ∣ s , a ) \mathcal{T}(s^{\prime}\mid s,a) and the observation model O ​ ( o ∣ s , a ) O(o\mid s,a) . The reward function R ​ ( s ) R(s) is defined only on terminal states and indicates task success.

At time step t t , the agent executes an action a t a{t} , after which the environment transitions to a latent state s t ∼ 𝒯 ( ⋅ ∣ s t − 1 , a t ) s{t}\sim\mathcal{T}(\cdot\mid s{t-1},a{t}) and returns an observation o t ∼ O ( ⋅ ∣ s t , a t ) o{t}\sim O(\cdot\mid s{t},a{t}) . During inference, neither the latent state s t s{t} nor the reward function is directly observable to the agent. This results in partial observability, which introduces several fundamental challenges, including latent state inference, task objective misalignment, and planning under uncertainty. In particular, inferring s t s{t} from the interaction history is difficult because the history often contains substantial redundant and noisy information. Moreover, not knowing s t s{t} creates a lot of uncertainty and further complicates the prediction of future situations in planning. In this work, we propose a novel framework to unifiedly resolve these challenges.

Knowledge Graph

Knowledge graphs (KGs) (Ehrlinger and Wöß 2016 ) represent information as graph-structured data, where nodes correspond to entities and edges encode relationships among them. KGs can be constructed from diverse sources, including structured databases, unstructured text, and other heterogeneous data modalities. Similar to traditional databases, graph databases support structured querying mechanisms, such as the Cypher query language (Francis et al. 2018 ) .

The relationship between a pair of nodes in a KG is commonly referred to as a triplet of subject, predicate, and object. In this work, we consider the following triplet forms:

• (entity 1, relationship, entity 2)

• (entity 1, attribute, boolean/value)

• (entity 1, verb in the past tense, entity 2)

which describe relationships, attributes and historic events of entities. A KG is denoted as 𝒢 = ( V , E ) \mathcal{G}=(V,E) where V V is a set of semantic vertices corresponding to entities in the environment, and E E is a set of semantic edges representing triplets in the above forms. Our framework has a retrieval-augmented generation (RAG) component which retrieves the relevant triplets from a KG and feeds them to LLM as context approximating the underlying state of the POMDP environment.

Twisted Sequential Monte Carlo

Twisted Sequential Monte Carlo (TSMC) (Doucet et al. 2001 ; Del Moral et al. 2006 ; Chopin et al. 2020 ; Zhao et al. 2024b ; Feng et al. 2025b ) is a probabilistic inference framework that incrementally guides particles toward high-probability regions of a target distribution σ \sigma through intermediate twisted distributions ψ t \psi_{t} . At each step, particles are propagated using a proposal distribution and resampled according to importance weights, allowing computational effort to focus on promising trajectories while reducing variance. In this work, to address uncertainty under partial observability, the slow-thinking module follows a TSMC-like structure, where LLMs are used to propose future states and observations and to score particles. For simplicity, full importance weights are not computed, since most LLM APIs do not provide access to log probabilities.

Sequential Monte Carlo (SMC) and its variants have been applied into planning and reinforcement learning by many previous work (Piché et al. 2018 ; Macfarlane et al. 2024 ; Abdulsamad et al. 2026 ) . Compared with other planning methods, such as Monte Carlo Tree Search (MCTS) (Chen et al. 2024 ) , SMC-style methods can better address the uncertainty of states and reduce computational complexity, since particles representing less-likely states will be discarded and not branched in large probability. However, in tree-search-based methods, every state will be branched and its child nodes will be evaluated. In this work, we develop a TSMC-style LLM-based planning method for slow-thinking module, where the state is approximated by related triplets retrieved from memory KG. As far as we know, this work is the first of doing so.

Figure 2: Diagram of NeSyFS framework. Knowledge graph (KG), representing the latent state of environment, is updated by fast-thinking module with new observation and retrieved by all the modules. Every reactive action is reflected. If "No", fast-thinking is called again with thought from reflection module. If "No" repeatedly for K K number of times, the slow-thinking module is initiated to search optimal action via planning, where memory KG is copied to initialize the simulation of future latent states and observations.

Methodology

In this section, we introduce the NeSyFS framework, which maintains a memory KG as a proxy for the latent state of the environment. The diagram of the proposed framework is shown in Figure 2 . Using triplets retrieved from the memory KG as a context, the fast-thinking module generates reactive actions, while the slow-thinking module performs uncertainty-aware deliberative planning. A reflection module evaluates the fast-thinking actions with respect to the task objective, where the KG-retrieved triplets facilitate reflection in the stepwise granularity. We will first introduce the fast-thinking module, including the memory KG. Then, reflection module will be presented with details. Finally, the slow-thinking module, including the proposed TSMC-style planning algorithm, is introduced.

Fast Thinking Module

We use a KG 𝒢 = ( V , E ) \mathcal{G}=(V,E) as working memory to represent the latent state of the environment (POMDP ℳ \mathcal{M} ). Instead of using the entire interaction history or its summary as unstructured context, this memory organizes environmental knowledge into structured triplets which describe attributes and relations of entities and past events, in the forms in Section Knowledge Graph .

Figure 3: Diagram of decision making in fast-thinking module.

At each time step t t , given the current observation o t o{t} and task description η \eta , the fast-thinking module first retrieves task-relevant triplets τ t \tau{t} from memory KG 𝒢 t \mathcal{G}{t} by calling the retrieval method ℛ ​ ( 𝒢 t , η ) \mathcal{R}(\mathcal{G}{t},\eta) . The details of ℛ \mathcal{R} are introduced in Appendix C . Then, τ t \tau{t} are then provided as contextual input to the LLM which is prompted to produce the next action for accomplishing task η \eta with observation o t o{t} . During this process, chain-of-thought (CoT) reasoning (Wei et al. 2022 ) is employed, making the fast-thinking action selection process analogous to the ReAct method (Yao et al. 2022b ) . This can be formally expressed as a t = L ​ L ​ M ​ ( P fast ; ℛ ​ ( 𝒢 t ) , o t , η ) a{t}=LLM(P^{\text{fast}};\mathcal{R}(\mathcal{G}{t}),o{t},\eta) , where P fast P^{\text{fast}} is the reactive decision making prompt template, 𝒢 t \mathcal{G}{t} is the KG representing current latent state. Diagram of decision making in the fast-thinking module is shown in Figure 3 .

Whenever a t a{t} is applied into the environment, the agent receives a new observation o t o{t} . The KG 𝒢 t \mathcal{G}{t} will be updated with o t o{t} and a t a{t} to produce 𝒢 t + 1 \mathcal{G}{t+1} to represent next state. This process is denoted as 𝒢 t + 1 = Update ​ ( 𝒢 t , o t , a t ) \mathcal{G}{t+1}=\text{Update}(\mathcal{G}{t},o{t},a{t}) , where new triplets from o t o{t} and a t a{t} will be added to 𝒢 t \mathcal{G}{t} , and outdated triplets in 𝒢 t \mathcal{G}{t} will be removed. The details of update process are presented in Appendix C .

Figure 4: Diagram of the proposed TSMC-style uncertainty-aware planning algorithm, where three particles are updated in the planning step t t , consisting of propagation, weight update and resampling processes. The red box shows the propagation process, where the prediction of KG at time t + 1 t+1 is essentially updating KG at t t with a t + 1 a{t+1} and o t + 1 o{t+1} . The blue box is the weight update process, where the blue circle represents the value of an updated weight. The green box represents the process of resampling. Every particle stores all the historic observations, actions, states (KG), and weights.

Reflection Module

Due to incomplete information about the underlying environment state, partial observability can introduce misalignment between the agent’s actions and the task objective, which has emerged as a major challenge for LLM agents (Fang et al. 2025 ; Kim et al. 2025 ; Chung et al. 2025 ) . To mitigate this issue, we introduce a prompt-based reflection module that detects misalignment between the actions produced by the fast-thinking component and the task objective. Unlike self-reflection methods (Shinn et al. 2023 ) , in which the agent introspects and explains its own behavior, the proposed reflection module adopts a third-person perspective to verify whether a candidate action aligns with the task objective. Our innovation here is to use the KG-retrieved triplets as context to facilitate reflection in the stepwise granularity, since context triplets provide sufficient state information for reflection.

The reflection can be expressed as Y ref = L ​ L ​ M ​ ( P ref ; a t , o t , ℛ ​ ( 𝒢 t ) , η ) Y^{\text{ref}}=LLM(P^{\text{ref}};a{t},o{t},\mathcal{R}(\mathcal{G}{t}),\eta) , where P ref P^{\text{ref}} is the prompt for refection, a t a{t} is the fast-thinking action, ℛ \mathcal{R} is the operation of KG retrieval introduced in previous section, η \eta is the task description, and output Y ref ∈ { Yes , No } Y^{\text{ref}}\in{\text{Yes},\text{No}} which indicates whether a t a_{t} aligns with η \eta or not. To mitigate the randomness of LLM output, we propose to adopt a self-consistency method (Wang et al. 2022b ) which draws n n samples of Y ref Y^{\text{ref}} from LLM and sets the final output Y ¯ ref \bar{Y}^{\text{ref}} as "Yes" only when 80 % 80% of Y ref Y^{\text{ref}} samples are "Yes". We use n = 5 n=5 in experiments.

As shown in Figure 2 , if the final reflection answer Y ¯ ref \bar{Y}^{\text{ref}} is "Yes", then a t a_{t} is applied to the environment. Otherwise, the thought of reflection module which produces "No" is sent back to the fast-thinking module and a new reactive action is produced by calling the fast-thinking module again. If Y ¯ ref \bar{Y}^{\text{ref}} is repeatedly not "Yes" for K K number of times, say K = 3 K=3 , the slow thinking module is activated.

Slow Thinking Module

As shown in Figure 2 , when the fast-thinking is unreliable, the agent transitions to the slow-thinking module, which performs deliberative planning by explicitly searching over candidate action sequences in a step-by-step manner. To handle the uncertainty arising from partial observability, we propose a novel neuro-symbolic uncertainty-aware planning algorithm which follows the high-level framework of twisted sequential Monte Carlo (TSMC) algorithm introduced in Section Twisted Sequential Monte Carlo .

The proposed planning algorithm is illustrated in Figure 4 . Specifically, the algorithm maintains a set of weighted particles that represent possible future observations, latent states (i.e., KGs), and actions over a planning horizon H H . To identify an optimal plan, these particles are guided toward high-probability regions of a target distribution σ \sigma , which corresponds to trajectories that successfully accomplish the task. At each planning step, every particle is propagated by predicting the next action, observation, and latent state (KG). The particle weights are then updated according to the likelihood of the newly predicted state achieving task progress. Finally, a resampling procedure based on the updated weights preserves particles that make meaningful task progress while filtering out less promising particles with a large probability.

To address the difficulty LLMs face in evaluating task progress (Setlur et al. 2025 ; Zheng et al. 2025 ) , particle weights are updated using binary LLM feedback indicating whether a given state transition makes task progress or not. Moreover, the concise state representation provided by KG can improve the correctness of LLM judgment. In addition, the resampling mechanism can make our approach more robust to LLM’s mistakes, as it still allows promising particles to survive even when they are temporarily misjudged by LLM.

Compared with previous tree-search planning methods, such as MCTS (Chen et al. 2024 ) , the proposed TSMC-style method is better at addressing state uncertainty and more computationally efficient. Previous methods are difficult at predicting next state or observation under partial observability, while the proposed method leverages memory KG to accurately predict next KG by updating current KG with new action and observation. In addition, in tree-search methods, every leaf node is expanded by predicting multiple next states as child nodes. However, in the proposed method, particles with poor task progress will be discarded and not be expanded with a large probability, hence reducing the computation complexity.

Definition. Denote N N as number of particles. Denote particle i i at the planning step τ \tau as 𝒑 τ i = ( 𝒐 ^ 1 : τ i , 𝓖 ^ 1 : τ i , 𝒂 ^ 1 : τ i , ω τ i ) \bm{p}{\tau}^{i}=(\hat{\bm{o}}^{i}{1:\tau},\hat{\bm{\mathcal{G}}}^{i}{1:\tau},\hat{\bm{a}}^{i}{1:\tau},\omega^{i}{\tau}) , where 𝒐 1 : τ i = ( o ^ 1 i , … , o ^ τ i ) \bm{o}^{i}{1:\tau}=(\hat{o}^{i}{1},\ldots,\hat{o}^{i}{\tau}) is the sequence of observations predicted by LLM until time step τ \tau , and same for 𝓖 ^ 1 : τ i \hat{\bm{\mathcal{G}}}^{i}{1:\tau} and 𝒂 ^ 1 : τ i \hat{\bm{a}}^{i}{1:\tau} .

Initialization. Denote t 0 t{0} as the time step when slow thinking is activated. For each particle, set the first observation, latent state and action as copies of the actual observation, state (KG), and action at time step t 0 t{0} , i.e. o ^ 1 i = o t 0 , 𝒢 ^ 1 i = 𝒢 t 0 , a ^ 1 i = a t 0 \hat{o}^{i}{1}=o{t{0}},\hat{\mathcal{G}}^{i}{1}=\mathcal{G}{t{0}},\hat{a}^{i}{1}=a{t{0}} for i = 1 , … , N i=1,\ldots,N , so that every particle starts from the same origin. Set the weight of each particle to be 1, i.e. ω 1 i = 1 \omega^{i}{1}=1 .

Propagation. As shown in the red box of Figure 4 , in each planning step τ \tau , every particle 𝒑 τ i \bm{p}{\tau}^{i} is propagated forward to generate a new particle 𝒑 τ + 1 i \bm{p}{\tau+1}^{i} by predicting the next action a ^ τ + 1 i \hat{a}{\tau+1}^{i} , observation o ^ τ + 1 i \hat{o}{\tau+1}^{i} , and latent state 𝒢 ^ τ + 1 i \hat{\mathcal{G}}_{\tau+1}^{i} one-by-one.

First, the next action is produced by an LLM-based policy, described as a ^ τ + 1 i = L ​ L ​ M ​ ( P fast ; ℛ ​ ( 𝒢 ^ τ i ) , o ^ τ i , η ) \hat{a}{\tau+1}^{i}=LLM(P^{\text{fast}};\mathcal{R}(\hat{\mathcal{G}}{\tau}^{i}),\hat{o}_{\tau}^{i},\eta) , where P fast P^{\text{fast}} is the prompt template used in the fast-thinking module, and η \eta is the task description.

Subsequently, for each particle i i , we need to predict the next observation o ^ τ + 1 i \hat{o}^{i}{\tau+1} caused by action a ^ τ + 1 i \hat{a}^{i}{\tau+1} . Specifically, task-relevant context triplets are retrieved from the current KG 𝒢 ^ τ i \hat{\mathcal{G}}{\tau}^{i} and used as state information, which is the same process as the retrieval ℛ ​ ( 𝒢 ^ τ i , η ) \mathcal{R}(\hat{\mathcal{G}}^{i}{\tau},\eta) described in the fast-thinking section. Then with these triplets and a ^ τ + 1 i \hat{a}^{i}{\tau+1} as input, LLM is prompted to predict the next observation o ^ τ + 1 i \hat{o}{\tau+1}^{i} . This can be formally written as o ^ τ + 1 = L ​ L ​ M ​ ( P obs ; ℛ ​ ( 𝒢 ^ τ i , η ) , a ^ τ + 1 i ) \hat{o}{\tau+1}=LLM(P^{\text{obs}};\mathcal{R}(\hat{\mathcal{G}}^{i}{\tau},\eta),\hat{a}_{\tau+1}^{i}) . In this prediction, the LLM is asked to infer plausible future observations using its commonsense knowledge and reasoning capabilities.

Then, o ^ τ + 1 i \hat{o}{\tau+1}^{i} is parsed into triplets which are incorporated into 𝒢 ^ τ i \hat{\mathcal{G}}{\tau}^{i} to produce the next latent state (KG) 𝒢 ^ τ + 1 i \hat{\mathcal{G}}{\tau+1}^{i} , following the process of updating KG introduced in the fast-thinking section, i.e. 𝒢 ^ τ + 1 i = Update ​ ( 𝒢 ^ τ i , o ^ τ + 1 i , a ^ τ + 1 ) \hat{\mathcal{G}}{\tau+1}^{i}=\text{Update}(\hat{\mathcal{G}}{\tau}^{i},\hat{o}{\tau+1}^{i},\hat{a}{\tau+1}) . The newly acquired triplets in updating process are denoted as τ new \tau{\text{new}} .

Weight Update. As shown in the blue box of Figure 4 , each particle i i is assigned a new weight ω τ + 1 i \omega{\tau+1}^{i} which is updated as ω τ + 1 i ← ω τ i ​ exp ⁡ ( λ ​ δ τ + 1 i ) \omega{\tau+1}^{i}\leftarrow\omega{\tau}^{i}\exp(\lambda\delta^{i}{\tau+1}) . Specifically, δ τ + 1 i \delta^{i}{\tau+1} is the task progress detector and is the LLM evaluation on whether the transition ( 𝒢 ^ τ i , a ^ τ i , 𝒢 ^ τ + 1 i ) (\hat{\mathcal{G}}{\tau}^{i},\hat{a}{\tau}^{i},\hat{\mathcal{G}}{\tau+1}^{i}) makes progress toward task completion or not. Formally, we can write δ τ + 1 i = L ​ L ​ M ​ ( P prog ; a ^ τ i , ℛ ​ ( 𝒢 ^ τ i ) , τ new , η ) \delta^{i}{\tau+1}=LLM(P^{\text{prog}};\hat{a}{\tau}^{i},\mathcal{R}(\hat{\mathcal{G}}^{i}{\tau}),\tau{\text{new}},\eta) , where P prog P^{\text{prog}} is the prompt template in the task progress evaluation, δ τ + 1 i ∈ { 0 , 1 } \delta^{i}{\tau+1}\in{0,1} , and τ new \tau{\text{new}} is obtained in the propagation process. In contrast to previous LLM-based evaluation methods (Zheng et al. 2025 ) , in our approach, the binary response of LLM and concise representation provided by context triplets can reduce hallucination and make LLM focus on task-related information in the input.

Resampling. The resampling step is to probabilistically keep particles with good task progress and discard particles without poor progress. At planning step τ \tau , the likelihood of a particle i i being kept is proportional to its weight ω τ i \omega_{\tau}^{i} .

Repeating the operations of propagation, weight update and resampling over the planning horizon H H , the remaining particles are close to successful trajectories with a large probability. This can be guaranteed by the theoretical properties of TSMC (Del Moral et al. 2006 ) . After planning step H H , uniformly sample a particle j ∼ Unif ​ ( [ 1 , … , N ] ) j\sim\text{Unif}([1,\ldots,N]) , and apply the first action in the particle j j (i.e. a ^ 1 j \hat{a}_{1}^{j} ) to the actual environment, as shown in Figure 2 . The theoretical justification is in Appendix C .

Experiments

Experimental Settings

Benchmarks. We benchmarked NeSyFS on three widely used text-based environments: ALFWorld (Shridhar et al. 2021 ) , WebShop (Yao et al. 2022a ) , and ScienceWorld (Wang et al. 2022a ) . ALFWorld evaluates embodied agents on household tasks, WebShop (Yao et al. 2022a ) emulates multi-step decision-making tasks in an online shopping website environment, and ScienceWorld (Wang et al. 2022a ) assesses procedural and scientific reasoning in educational scenarios. For evaluation, ALFWorld uses binary task success, while WebShop and ScienceWorld provide dense reward signals, enabling evaluation based on both success rate and average reward, calculated as the mean reward across all tasks. Additional benchmark details are presented in Appendix D .

Table 1: Performance comparison of NeSyFS with ReAct, Reflexion, RAFA and SwiftSage across ALFWorld, WebShop and ScienceWorld. SR and AR denote success ratio and average reward, respectively.

Model Method ALFWorld WebShop ScienceWorld Average

SR AR SR AR SR

GPT-5-mini ReAct 71.2 71.2 42.5 42.5 32.7 32.7 65.1 65.1 28.3 28.3 44.1 44.1

Reflexion 75.6 75.6 49.5 49.5 35.7 35.7 69.7 69.7 31.5 31.5 47.6 47.6

ABBEL 72.1 72.1 43.2 43.2 33.3 33.3 65.9 65.9 29.5 29.5 44.9 44.9

RAFA 73.5 73.5 40.1 40.1 27.3 27.3 70.2 70.2 34.5 34.5 45.1 45.1

SwiftSage 76.7 76.7 52.3 52.3 38.2 38.2 72.1 72.1 35.7 35.7 51.0 51.0

NeSyFS 91.1 \bm{91.1} 61.2 \bm{61.2} 51.3 \bm{51.3} 82.2 \bm{82.2} 61.2 \bm{61.2} 63.5 \bm{63.5}

GPT-5 ReAct 78.2 78.2 49.6 49.6 38.7 38.7 72.2 72.2 35.5 35.5 50.8 50.8

Reflexion 80.3 80.3 52.3 52.3 40.2 40.2 75.7 75.7 38.9 38.9 53.1 53.1

ABBEL 77.6 77.6 51.2 51.2 39.8 39.8 73.9 73.9 36.0 36.0 51.1 51.1

RAFA 83.2 83.2 48.1 48.1 35.3 35.3 76.3 76.3 42.7 42.7 53.7 53.7

SwiftSage 81.9 81.9 56.3 56.3 41.2 41.2 75.3 75.3 40.2 40.2 54.4 54.4

NeSyFS 93.6 \bm{93.6} 69.4 \bm{69.4} 63.3 \bm{63.3} 86.1 \bm{86.1} 69.4 \bm{69.4} 75.3 \bm{75.3}

Llama-3.3-70B ReAct 79.6 79.6 45.6 45.6 37.1 37.1 69.2 69.2 33.1 33.1 49.9 49.9

Reflexion 81.2 81.2 47.3 47.3 38.9 38.9 73.1 73.1 36.9 36.9 52.3 52.3

ABBEL 78.6 78.6 45.1 45.1 36.9 36.9 70.1 70.1 32.5 32.5 49.3 49.3

RAFA 82.3 82.3 47.2 47.2 37.1 37.1 75.1 75.1 39.2 39.2 52.9 52.9

SwiftSage 80.6 80.6 51.2 51.2 39.9 39.9 76.1 76.1 41.3 41.3 53.9 53.9

NeSyFS 89.6 \bm{89.6} 68.1 \bm{68.1} 62.5 \bm{62.5} 85.2 \bm{85.2} 67.1 \bm{67.1} 73.0 \bm{73.0}

Agent Models. In the evaluations, we use GPT-5, GPT-5-mini (Singh et al. 2025 ) and Llama-3.3-70B-Instruct (Grattafiori et al. 2024 ) as the underlying models. GPT-5 variants serve as proprietary models, while Llama-3.3 represents the open-source counterpart.

In the following, we first conduct experiments to compare the overall framework against some previous representative methods. Then, the effect of reflection module is investigated empirically, independently of the slow-thinking module. Finally, the performance of planning algorithm in the slow-thinking module is specifically evaluated in Appendix by removing the reflection module. Every result reported here is an average of three random seeds.

Overall Evaluation

Baseline. Some representative decision making methods of LLM agent are selected as baselines. Details of each baseline are introduced in Appendix D .

• ReAct (Yao et al. 2022b ) : This method reasons about next action by using CoT (Wei et al. 2022 ) , laying the foundation of LLM decision making.

• Reflexion (Shinn et al. 2023 ) : This method reflects every failed trajectory and helps LLMs improve after each failed attempt.

• ABBEL (Lidayan et al. 2025 ) : It uses LLM to summarize observation and full interaction history to infer the belief state.

• RAFA (Liu et al. 2023 ) : At each time step, the agent first conducts a short-horizon planning through tree search, executes the first action of the plan, and then replans at next step.

• SwiftSage (Lin et al. 2023 ) : In this fast-slow thinking framework, the Swift module represents fast and intuitive thinking, and the Sage module plans and grounds subgoals to emulate deliberate thought processes.

Results. Table 1 presents the performance comparison between NeSyFS and several representative baselines, including ReAct, Reflexion, ABBEL, RAFA, and SwiftSage, across ALFWorld, WebShop, and ScienceWorld. The results show that NeSyFS consistently and significantly outperforms all baseline methods. In particular, NeSyFS achieves nearly a 100% improvement over ReAct and Reflexion, which serve as the foundation of many state-of-the-art LLM agents, highlighting the effectiveness of the proposed KG-based memory, reflection, and slow-thinking modules.

Furthermore, the substantial improvement over summarization-based method, such as ABBEL, suggests that KG-based contextual representations provide more effective state representations for decision making than summarization-based belief states, since the summarization of history may lose critical information for belief updates. RAFA performs short-horizon planning at each time step through tree search, a mechanism conceptually related to our slow-thinking module. The superior performance of NeSyFS over RAFA therefore underscores the importance of integrating KG-based memory and reflection mechanisms in addition to planning. SwiftSage represents a typical fast–slow thinking framework, and the advantage of NeSyFS further demonstrates the benefits of KG-based memory in enhancing the synergy between fast and slow reasoning processes. Comparisons with previous KG-augmented LLM-agent methods (Agarwal et al. 2025 ; Anokhin et al. 2025 ) are provided separately in the next section.

KG-provided Context and Reflection

In this section, we first evaluate the correctness of the KG-augmented reflection module, and then demonstrate the advantage and effect of KG-retrieved context triplets in the overall task-completion evaluations.

The reflection module of NeSyFS leverages KG-retrieved triplets as contextual information. In the first part of experiment, to evaluate the effect of KG-provided context in reflection, we compare the correctness of our reflection module against baselines that use the entire interaction history and LLM-summarized belief state as context, which are short as "history" and "belief", respectively. The proposed reflection method is short as "KG". All the methods are evaluated on a set of randomly-collected trajectories in which the correctness of each action is manually annotated.

We adopt two evaluation metrics. The first is Total Detection Errors (TDE), defined as the sum of false positives (misaligned actions that are not detected) and false negatives (correct actions incorrectly classified as misaligned). TDE measures the overall number of reflection errors. The second metric is Effective Reliability (ER), defined as T ​ P − F ​ P T ​ P + F ​ P \frac{TP-FP}{TP+FP} , where T ​ P TP and F ​ P FP denote the numbers of true positives and false positives, respectively. Hence, ER evaluates the reliability of actions that are approved by the reflection module.

Table 2: Comparison of reflection with different representations of context.

Model Method ALFWorld WebShop ScienceWorld

TDE ER TDE ER TDE ER

GPT-5 History 57 0.55 87 0.51 95 0.53

Belief 52 0.59 89 0.50 89 0.55

KG 42 0.71 61 0.66 77 0.65

GPT-5-mini History 82 0.45 113 0.43 135 0.42

Belief 77 0.41 110 0.42 125 0.45

KG 58 0.65 72 0.58 91 0.59

The performance comparison is shown in Table 2 . We can see that the reflection method using KG-provided context significantly outperforms that of using interaction history or LLM-summarized belief state as context. This is because KG-retrieved triplets can provide concise state representation, making the LLM focus on decision-making-related information in the prompt when reflecting. Notably, the ER of "KG" method is significantly higher than baselines, meaning that KG-provided context could prevent more wrong actions from being applied into the environment. "History" performs worse since the noisy and redundant information in the interaction history can distract LLM. The problem of "Belief" is that the LLM-based summarization can lose important information.

(a) GPT-5-mini

(b) GPT-5 Figure 5: Performance comparison of different reflection methods.

In the second part of the experiment, we empirically investigate the effect of KG-provided context within the fast-thinking and reflection modules by comparing different methods of forming the context to approximate the underlying environment state. Both GPT-5 and GPT-5-mini are used as the underlying LLMs in these evaluations. The evaluation metric, corresponding to y-axis in the plots, is the task success rate.

The methods “History Fast”, “Belief Fast”, and “KG Fast” use the full interaction history, an LLM-summarized belief state, and KG-retrieved triplets, respectively, as context for action selection in the fast-thinking module, without employing any reflection module or slow-thinking module. In contrast, “History Ref”, “Belief Ref”, and “KG Ref” incorporate the reflection module while using the same three forms of context representations. Specifically, in the fast-thinking and reflection modules, the KG is retrieved and updated using the same procedures as those employed in NeSyFS, as described in Section Fast Thinking Module . Besides, the belief state is updated using the same method as ABBEL introduced in Appendix D . No slow-thinking module is used in any of these evaluations here. Notably, the “Fast” method are conceptually equivalent to previous KG-augmented LLM agents (Agarwal et al. 2025 ; Anokhin et al. 2025 ) .

As illustrated in Figure 5 , incorporating the reflection module consistently improves performance across all benchmarks. Moreover, the use of KG-provided context leads to additional performance gains, demonstrating both the importance of reflection in mitigating misalignment with task objectives and the effectiveness of the symbolic, structured representations provided by the KG.

Conclusion

In this work, we propose a neuro-symbolic framework which addresses the challenges introduced by partial observability in a unified manner. Under partial observability, the agent may have difficulties on latent state inference, task objective misalignment, and planning under uncertainty. In NeSyFS, the latent state of the environment is represented and maintained as a memory KG, and triplets retrieved from KG are used as context in every module of NeSyFS. In addition, a KG-augmented reflection module is proposed to address the misalignment of the task objective. Besides, a neuro-symbolic TSMC-style planning algorithm is proposed to tackle the uncertainty in observation prediction and task progress evaluation. We empirically validate the proposed framework and its modules in three popular benchmarks.