Build a Multi-Agent Supervisor with LangGraph

Below roughly 6, 8 tools, with a well-written system prompt and clear tool descriptions, a single agent is usually simpler, cheaper, and just as accurate, the supervisor pattern adds an extra LLM call per query for the routing decision. The multi-agent win kicks in around 10+ tools, or whenever tools cluster into clearly-different domains (research, math, writing, code, retrieval). If your tools are all variants of 'query one of five databases', a single agent is fine; if they span modalities, specialists start to pay off.

In the lab it's a small LLM call with a classification-style prompt, 'Given the user's query, return exactly one of {researcher, calculator, writer}', parsed and written to state['next_agent']. That's cheap and flexible but stochastic; for latency-critical paths you can replace it with a keyword classifier, a small fine-tuned model, or even a regex-based pre-filter that only falls through to the LLM for ambiguous queries. The conditional edge after the router reads state['next_agent'] and dispatches to the right specialist node, so swapping the classifier is a one-node change.

A TypedDict with the fields every node needs to read or update, typically at minimum messages: list[BaseMessage] (appended by every node) and routing fields like next_agent: str. Each node returns a dict of updates and LangGraph merges them according to each field's annotated reducer (add_messages for the messages list, overwrite for scalar fields). The lab keeps the state small and typed on purpose so the graph's control flow stays legible; production systems often carry user id, thread id, and per-specialist scratchpads in the same state object.

Because 'write a three-paragraph summary' and 'find five facts from the knowledge base' have different success criteria and need different prompts. The researcher is tuned for breadth and recall; the writer is tuned for structure, concision, and style. Mixing them creates a prompt that's mediocre at both. The lab specifically has you add the writer as a third node to make the scaling property concrete, adding a specialist is strictly additive: one new tool, one new agent, one new router label, zero changes to existing nodes.

Two options. The simple one is have the router emit a sequence of specialists instead of one, and loop: researcher returns facts, writer reads the messages state and drafts the report. The more robust one is a reducer-style graph where each specialist writes into a shared scratchpad and a synthesizer node at the end merges everything. The lab sticks with single-specialist routing to keep the pattern crisp; once you're comfortable, extending to multi-step routing is a small modification to the conditional edge logic and the state shape.

Pathological looping, specialist A returns an answer the router doesn't recognise as final, routes back into specialist B, specialist B produces output that re-routes to A, and so on. LangGraph's recursion limit saves you in principle but the correct fix is to make the END transition explicit in each specialist (the ReAct loop terminates when the LLM returns text without tool calls) and to add a 'max specialist hops' counter to state. The reflection step pushes you toward instrumenting exactly this so you can detect and cap it before it eats your NIM quota.