Agent Planning: ReAct vs CoT vs Tree of Thoughts (NCP-AAI)

Planning -- the ability to break down complex goals into executable steps -- is what separates advanced agentic AI systems from simple chatbots. The NVIDIA NCP-AAI certification heavily emphasizes planning strategies, as they determine an agent's capability to solve multi-step problems, reason about consequences, and optimize action sequences. This comprehensive guide covers every planning paradigm tested on the NCP-AAI exam: the three foundational reasoning strategies (Chain-of-Thought, ReAct, and Tree of Thoughts), five classical planning approaches (forward, backward, HTN, partial-order, and continual), advanced algorithms (A* and MCTS), NVIDIA NeMo Agent Toolkit planning modules, multi-agent planning patterns, and common exam traps you need to avoid.

Why Planning Matters for Agentic AI

Planning enables agents to:

• Decompose complex tasks into manageable sub-tasks
• Reason about action sequences before execution
• Anticipate obstacles and plan contingencies
• Optimize for goals (shortest path, lowest cost, highest success rate)
• Handle ambiguous or underspecified requests
• Allocate resources efficiently across parallel workstreams
• Adapt dynamically when circumstances change

NCP-AAI Coverage:

• Agent Design and Cognition domain (15%): Planning algorithms, reasoning patterns, goal management
• Agent Development domain (15%): Implementing planning mechanisms, framework selection
• Agent Architecture domain (15%): Choosing appropriate planning strategies, multi-agent coordination

The Planning Challenge

Without planning, agents exhibit three critical failure modes:

• Myopic behavior: Short-sighted decisions without considering future consequences
• Action thrashing: Inefficient trial-and-error without strategic thinking
• Goal confusion: Losing track of the original objective in multi-step tasks

Example -- Flight Booking Without vs. With Planning:

Task: "Book a flight to Paris for next week"

Without Planning (Bad):
Agent: "What dates work for you?"
User: "Monday to Friday"
Agent: "Let me search flights..."
Agent: "Oh, I need your departure city. What city?"
User: "San Francisco"
Agent: "Searching... Oh, I need your budget. What's your budget?"
→ Inefficient, poor user experience (3 round trips)

With Planning (Good):
Agent: "To book your flight, I need:
  1. Departure city
  2. Travel dates
  3. Budget range
  4. Seating preference
Can you provide these details?"
→ Strategic, efficient information gathering (1 round trip)

Core Planning Capabilities Tested on NCP-AAI

The exam tests your understanding of five core planning capabilities:

• Task decomposition: Breaking complex requests into ordered subtasks
• Multi-step orchestration: Sequencing actions with correct dependencies
• Conditional branching: Adapting plans based on runtime conditions
• Error recovery: Replanning when tasks fail (fallback strategies)
• Goal optimization: Finding efficient paths to objectives under constraints

Chain-of-Thought (CoT) Prompting

Definition: Chain-of-Thought prompting elicits step-by-step reasoning from LLMs by showing examples or instructing the model to "think through" problems. Introduced by Wei et al. (2022) in "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," CoT demonstrated that prompting a 540B-parameter model (PaLM) with just eight chain-of-thought exemplars achieved state-of-the-art accuracy on the GSM8K benchmark of math word problems.

How CoT Works

CoT decomposes complex reasoning into explicit intermediate steps, making the model's thought process transparent and verifiable. Rather than jumping from question to answer, the model generates a reasoning trace that walks through each logical step.

Basic CoT (Few-Shot)

Provide examples of step-by-step reasoning to teach the model the format:

from langchain.prompts import PromptTemplate
from langchain_openai import ChatOpenAI

few_shot_cot_template = """
Example 1:
Question: A bakery makes 48 cupcakes. If they pack 6 cupcakes per box, how many boxes do they need?
Reasoning:
- Total cupcakes: 48
- Cupcakes per box: 6
- Calculation: 48 / 6 = 8
Answer: 8 boxes

Example 2:
Question: If a car travels at 60 mph for 2.5 hours, how far does it travel?
Reasoning:
- Speed: 60 miles per hour
- Time: 2.5 hours
- Formula: Distance = Speed x Time
- Calculation: 60 x 2.5 = 150
Answer: 150 miles

Now solve this:
Question: {question}
Reasoning:
"""

llm = ChatOpenAI(model="gpt-4", temperature=0)
chain = PromptTemplate(input_variables=["question"], template=few_shot_cot_template) | llm

result = chain.invoke({
    "question": "If a store sells 15 items per hour and is open 8 hours per day, how many items are sold in a week?"
})

# Output:
# - Items per hour: 15
# - Hours per day: 8
# - Items per day: 15 x 8 = 120
# - Days per week: 7
# - Items per week: 120 x 7 = 840
# Answer: 840 items per week

Zero-Shot CoT

No examples needed -- just append "Let's think step by step" to the prompt. Kojima et al. (2022) showed in "Large Language Models are Zero-Shot Reasoners" that this simple addition dramatically improves reasoning performance without any exemplars:

zero_shot_cot_prompt = """
Question: {question}

Let's think step by step:
"""

# Results from Kojima et al. (2022) using text-davinci-002:
# MultiArith: 17.7% → 78.7% accuracy
# GSM8K:      10.4% → 40.7% accuracy
# Similar improvements observed with PaLM 540B

The versatility of this single prompt across diverse reasoning tasks -- arithmetic, symbolic, commonsense, and logical -- hints at untapped zero-shot cognitive capabilities in large language models.

Self-Consistency CoT

Generate multiple CoT reasoning paths and select the answer that appears most frequently (majority vote). This reduces the impact of any single flawed reasoning chain:

class SelfConsistencyCoT:
    def __init__(self, llm, num_samples=5):
        self.llm = llm
        self.num_samples = num_samples

    def solve(self, question):
        answers = []
        for _ in range(self.num_samples):
            # Generate with temperature > 0 for diverse reasoning paths
            response = self.llm.predict(
                f"Question: {question}\nLet's think step by step:",
                temperature=0.7
            )
            answer = self.extract_answer(response)
            answers.append(answer)

        # Majority vote
        from collections import Counter
        most_common = Counter(answers).most_common(1)[0][0]
        return most_common

# Typically improves accuracy by 5-15% over single-pass CoT

CoT Strengths and Limitations

CoT Variants Summary

The NCP-AAI exam may present scenarios where you need to choose between CoT variants. Here is a quick reference:

Key Research Results to Remember:

• Wei et al. (2022): Few-shot CoT with PaLM 540B achieved state-of-the-art on GSM8K math benchmarks, demonstrating that reasoning emerges in sufficiently large models with appropriate prompting.
• Kojima et al. (2022): Zero-shot CoT ("Let's think step by step") improved MultiArith accuracy from 17.7% to 78.7% with InstructGPT, showing that no exemplars are needed to unlock reasoning capabilities.
• Wang et al. (2022): Self-Consistency improved CoT accuracy by 5-15% across benchmarks by sampling multiple reasoning paths and taking the majority vote, addressing the fragility of single-chain reasoning.

When CoT Fails: Understanding Limitations

CoT has well-documented failure modes that the NCP-AAI exam may test:

1. Faithful but wrong reasoning: The model can generate a logical-looking chain of reasoning that arrives at the wrong answer. The steps appear sound, but a subtle error early in the chain propagates forward.

2. Overthinking simple problems: For straightforward factual lookups or pattern matching, CoT adds unnecessary tokens and latency without improving accuracy. Use DIRECT mode for these.

3. No external grounding: CoT operates entirely on the model's internal knowledge. If the information is outdated, incomplete, or hallucinated, the entire reasoning chain is built on a faulty foundation. This is precisely why ReAct was developed.

4. Length sensitivity: Very long reasoning chains (10+ steps) can lose coherence, with the model forgetting earlier constraints or introducing contradictions. For truly complex problems, hierarchical decomposition (HTN) may be more appropriate.

ReAct: Reasoning + Acting

Definition: ReAct (Reasoning and Acting) interleaves reasoning traces with action execution, allowing agents to dynamically adjust plans based on environment feedback. Introduced by Yao et al. (2022) in "ReAct: Synergizing Reasoning and Acting in Language Models," the framework addresses a key limitation of CoT: while chain-of-thought uses only internal representations, ReAct grounds reasoning in real-world observations by interleaving thought with action.

Why ReAct Matters

The core insight of ReAct is that reasoning and acting are complementary:

• Reasoning traces help the model induce, track, and update action plans, as well as handle exceptions
• Actions allow the model to interface with external sources (knowledge bases, APIs, environments) to gather additional information

Yao et al. evaluated ReAct on four benchmarks -- HotPotQA, Fever, ALFWorld, and WebShop -- and found that ReAct outperforms vanilla action generation while being competitive with CoT. The best results came from combining ReAct with CoT, using both internal knowledge and externally obtained information.

The ReAct Loop

Thought: [Reasoning about what to do next]
Action: [Tool/function to execute]
Action Input: [Arguments for the tool]
Observation: [Result from executing the action]
... (repeat Thought/Action/Observation)
Thought: I now know the final answer
Final Answer: [Response to user]

ReAct Implementation

from langchain.agents import create_react_agent, AgentExecutor
from langchain.tools import Tool
from langchain_openai import ChatOpenAI

# Define tools
def search_web(query: str) -> str:
    """Search the web for information"""
    return f"Search results for '{query}': ..."

def calculator(expression: str) -> str:
    """Evaluate mathematical expressions"""
    try:
        return str(eval(expression))
    except Exception as e:
        return f"Error: {e}"

tools = [
    Tool(name="Search", func=search_web, description="Search the web for current information"),
    Tool(name="Calculator", func=calculator, description="Perform mathematical calculations")
]

# ReAct prompt template
react_prompt = """
Answer the following question as best you can. You have access to the following tools:

{tools}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought: {agent_scratchpad}
"""

# Create agent
llm = ChatOpenAI(model="gpt-4", temperature=0)
agent = create_react_agent(llm, tools, react_prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True, max_iterations=10)

# Execute
result = executor.invoke({"input": "What is the population of Tokyo multiplied by 2?"})

Execution Trace:

Thought: I need to find Tokyo's population, then multiply by 2
Action: Search
Action Input: "Tokyo population 2024"
Observation: Tokyo's population is approximately 14 million

Thought: Now I need to multiply 14 million by 2
Action: Calculator
Action Input: "14000000 * 2"
Observation: 28000000

Thought: I now know the final answer
Final Answer: The population of Tokyo (14 million) multiplied by 2 is 28 million.

ReAct Variants

1. ReAct with Self-Correction

The agent detects failures and adjusts its approach:

react_selfcorrect_prompt = """
... (standard ReAct format)

If an action fails or returns unexpected results, reconsider your approach:
Thought: That didn't work as expected. Let me try a different approach.
Action: [Alternative action]
...
"""

# Example execution trace:
# Thought: I'll search for "Tokyo population"
# Action: Search
# Action Input: "Tokyo population"
# Observation: Error: Too many results, be more specific
#
# Thought: That didn't work. Let me be more specific with the year.
# Action: Search
# Action Input: "Tokyo population 2024 census"
# Observation: Tokyo's population is approximately 14 million
# → Self-correction leads to success

2. ReAct with Reflection

The agent evaluates its own reasoning quality after task completion:

react_reflection_prompt = """
... (standard ReAct)

After completing the task, reflect:
Reflection: [Evaluate the quality of your reasoning and actions]
Improvements: [What could be done better next time]
"""

# Example:
# Final Answer: 28 million
#
# Reflection: My approach was effective -- I systematically gathered
# information and performed calculations. The search query could
# have been more precise initially.
#
# Improvements: Include the year in initial search queries to
# avoid ambiguity. Consider verifying with a second source.

3. ReAct + CoT Hybrid

Combine internal CoT reasoning with external ReAct actions for best results. In the original paper, Yao et al. found that combining ReAct with CoT outperformed either approach alone on HotPotQA and Fever benchmarks. The hybrid approach works by first attempting CoT internal reasoning, then switching to ReAct when the model detects that its internal knowledge is insufficient or outdated:

class ReActCoTHybrid:
    def __init__(self, llm, tools):
        self.llm = llm
        self.tools = tools

    def solve(self, question):
        # Step 1: Attempt CoT internal reasoning
        cot_response = self.llm.predict(f"""
        Question: {question}

        First, reason about what you already know (internal knowledge).
        Rate your confidence in your knowledge on a scale of 1-10.

        Internal reasoning:
        """)

        confidence = self.extract_confidence(cot_response)

        if confidence >= 8:
            # High confidence: use CoT answer directly (saves tool calls)
            return self.llm.predict(f"""
            Based on this reasoning: {cot_response}
            Provide the final answer:
            """)
        else:
            # Low confidence: switch to ReAct for external verification
            return self.react_agent.run(f"""
            Question: {question}
            Initial reasoning (may be incomplete): {cot_response}
            Verify and complete this answer using available tools.
            """)

# This pattern:
# - Saves tool calls when internal knowledge is sufficient
# - Falls back to grounded actions when knowledge gaps exist
# - Combines the speed of CoT with the accuracy of ReAct

This hybrid pattern is particularly valuable in production because it reduces API costs. Many agent queries can be answered with internal knowledge alone (saving tool call latency and cost), while complex or factual queries automatically escalate to tool-augmented reasoning. On the NCP-AAI exam, questions about optimizing agent costs while maintaining accuracy often point to this hybrid approach.

ReAct Advantages for NCP-AAI

1. Grounded in reality: Actions provide real feedback, preventing hallucinations
2. Transparent reasoning: Thought traces are interpretable and debuggable
3. Dynamic adaptation: Can adjust strategy based on observations
4. Tool integration: Natural fit for function calling and API interactions
5. Explicit and traceable: Unlike black-box planning, every decision is logged

NCP-AAI Exam Focus: ReAct is the default planning strategy for production agents due to its balance of reasoning and action. It is the most heavily tested planning framework on the exam.

ReAct Limitations

Tree of Thoughts (ToT): Exploring Multiple Reasoning Paths

Definition: Tree of Thoughts generates multiple reasoning paths (branches), evaluates them, and selects the most promising direction -- enabling search-based planning. Introduced by Yao et al. (2023) in "Tree of Thoughts: Deliberate Problem Solving with Large Language Models" (NeurIPS 2023), ToT generalizes CoT by allowing LMs to explore coherent units of text ("thoughts") as intermediate problem-solving steps, with the ability to look ahead, evaluate, and backtrack.

Key Result

On the Game of 24 benchmark, GPT-4 with standard CoT prompting solved only 4% of tasks, while ToT achieved a 74% success rate -- a dramatic improvement demonstrating the power of deliberate exploration over linear reasoning.

ToT Concepts

1. Thought Decomposition -- Break the problem into intermediate steps (thoughts), where each thought is a coherent unit of reasoning.

2. Thought Generation -- Generate multiple candidate thoughts at each step using the LLM.

3. State Evaluation -- Evaluate how promising each thought path is (how likely it leads to a correct solution). The LLM itself can serve as the evaluator, scoring each path on a numeric scale.

4. Search Algorithm -- Navigate the tree of possibilities:

• Breadth-First Search (BFS): Explore all options at each level before going deeper
• Depth-First Search (DFS): Explore one path deeply before backtracking
• Best-First Search: Prioritize the most promising paths using evaluation scores

ToT Implementation

from langchain_openai import ChatOpenAI

class TreeOfThoughts:
    def __init__(self, llm, max_depth=3, branching_factor=3):
        self.llm = llm
        self.max_depth = max_depth
        self.branching_factor = branching_factor

    def generate_thoughts(self, problem, current_state, depth):
        """Generate multiple candidate next thoughts"""
        prompt = f"""
        Problem: {problem}
        Current reasoning: {current_state}

        Generate {self.branching_factor} different possible next steps
        in solving this problem. Format each as a numbered option:
        """
        response = self.llm.predict(prompt)
        thoughts = self._parse_thoughts(response)
        return thoughts

    def evaluate_thought(self, problem, thought_sequence):
        """Evaluate how promising a thought sequence is (0-10)"""
        prompt = f"""
        Problem: {problem}
        Reasoning so far: {thought_sequence}

        On a scale of 0-10, how likely is this reasoning path
        to lead to a correct solution?
        Consider:
        - Logical coherence
        - Progress toward the goal
        - Avoiding dead ends

        Score (0-10):
        """
        response = self.llm.predict(prompt)
        score = float(response.strip())
        return score

    def _breadth_first_search(self, problem):
        """Explore all paths level by level"""
        queue = [("", 0)]  # (thought_sequence, depth)
        best_path = None
        best_score = -1

        while queue:
            current_state, depth = queue.pop(0)

            if depth >= self.max_depth:
                score = self.evaluate_thought(problem, current_state)
                if score > best_score:
                    best_score = score
                    best_path = current_state
                continue

            thoughts = self.generate_thoughts(problem, current_state, depth)
            for thought in thoughts:
                new_state = current_state + "\n" + thought
                queue.append((new_state, depth + 1))

        return best_path, best_score

    def _depth_first_search(self, problem, current_state="", depth=0, threshold=3):
        """Explore one path deeply, backtrack if score is too low"""
        if depth >= self.max_depth:
            score = self.evaluate_thought(problem, current_state)
            return current_state, score

        thoughts = self.generate_thoughts(problem, current_state, depth)
        best_path = None
        best_score = -1

        for thought in thoughts:
            new_state = current_state + "\n" + thought
            # Prune: skip paths that score below threshold
            mid_score = self.evaluate_thought(problem, new_state)
            if mid_score < threshold:
                continue  # Backtrack

            path, score = self._depth_first_search(
                problem, new_state, depth + 1, threshold
            )
            if score > best_score:
                best_score = score
                best_path = path

        return best_path, best_score

    def solve(self, problem, algorithm="bfs"):
        """Solve problem using Tree of Thoughts"""
        if algorithm == "bfs":
            best_path, score = self._breadth_first_search(problem)
        else:
            best_path, score = self._depth_first_search(problem)

        # Generate final answer from best path
        final_prompt = f"""
        Problem: {problem}
        Best reasoning path found:
        {best_path}

        Based on this reasoning, provide the final answer:
        """
        answer = self.llm.predict(final_prompt)
        return answer, best_path, score

# Usage
llm = ChatOpenAI(model="gpt-4", temperature=0.7)
tot = TreeOfThoughts(llm, max_depth=3, branching_factor=3)

problem = "Design a microservices architecture for an e-commerce platform."
answer, reasoning, score = tot.solve(problem)

ToT Execution Example

Problem: "Plan a 3-day trip to New York City on a $1000 budget"

Root: "Plan NYC trip, 3 days, $1000"
│
├── Branch 1: "Focus on free attractions (museums, parks, walking tours)"
│   ├── Branch 1.1: "Budget hotel ($100/night), subway pass"
│   │   └── Branch 1.1.1: "Day 1: Central Park + Met, Day 2: Brooklyn
│   │       Bridge + 9/11 Memorial, Day 3: High Line + Chelsea Market"
│   │       [Score: 8/10]
│   ├── Branch 1.2: "Hostel ($50/night), more budget for activities"
│   └── Branch 1.3: "Airbnb in Queens ($80/night), authentic experience"
│
├── Branch 2: "Prioritize iconic paid attractions"
│   ├── Branch 2.1: "Buy CityPass ($140), budget accommodation"
│   │   └── Branch 2.1.1: "Day 1: ESB + Top of Rock, Day 2:
│   │       Statue of Liberty..." [Score: 7/10]
│   └── Branch 2.2: "Focus on 2-3 major attractions, skip tourist traps"
│
└── Branch 3: "Cultural experience (Broadway, dining, neighborhoods)"
    ├── Branch 3.1: "One Broadway show ($150), street food"
    │   └── Branch 3.1.1: "Day 1: Broadway + Times Square, Day 2:
    │       Chinatown + Little Italy..." [Score: 6/10]
    └── Branch 3.2: "Skip Broadway, focus on food tours"

Best Path Selected: Branch 1.1.1 (Score: 8/10)
Reasoning: Maximizes experiences within budget by focusing on free
attractions while maintaining comfort with a budget hotel.

ToT Cost Analysis

The computational cost of ToT grows exponentially with depth and branching factor: