Multi-agent orchestration is a routing problem

One agent with many tools, or many agents

An agent with many tools keeps one conversation. On every turn the model sees the same history and picks the next call. A question that needs two shapes of work, such as looking up an order and then explaining a policy, still works, because the model can chain calls without leaving the context.

Orchestration begins when you stop trusting the model to hold the whole plan. The workflow below puts a planner in front: it turns the question into tasks, and each task is dispatched on its own. The task list leaves the planner at backend/app/lg_agent/kg_sub_graph/agentic_rag_agents/components/planner/node.py:56, where the output is written as "tasks": planner_output.tasks.

After that point nothing is shared except the task object. backend/app/lg_agent/kg_sub_graph/agentic_rag_agents/workflows/multi_agent/edges.py:61-72 returns a List[Send], one per task, each carrying only question and parent_task. That is the whole mechanism: tasks in, one dispatch per task, and no agent-to-agent conversation.

The results are merged by a summariser rather than by the tasks agreeing with each other. backend/app/lg_agent/kg_sub_graph/agentic_rag_agents/workflows/multi_agent/multi_tool.py:157-160 wires every tool branch into summarize and then into final_answer.

Three points that decide the shape

The implementation, in five excerpts

The workflow is one factory that assembles a LangGraph state graph. All excerpts below are verbatim from the Service Copilot backend, with line numbers so you can read the rest in context.

The factory signature already carries the two decisions that matter for reliability: a maximum attempt count, and whether a failing query may still be executed on the last attempt. Both default to the conservative choice: three attempts, no blind execution.

def create_multi_tool_workflow(
    llm: BaseChatModel,
    graph: Neo4jGraph,
    tool_schemas: List[type[BaseModel]],
    predefined_cypher_dict: Dict[str, str],
    cypher_example_retriever: BaseCypherExampleRetriever,
    scope_description: Optional[str] = None,
    llm_cypher_validation: bool = True,
    max_attempts: int = 3,
    attempt_cypher_execution_on_final_attempt: bool = False,
    default_to_text2cypher: bool = True,
) -> CompiledStateGraph:

main_graph_builder.add_edge(START, "guardrails")
main_graph_builder.add_conditional_edges(
    "guardrails",
    guardrails_conditional_edge,
)
main_graph_builder.add_conditional_edges(
    "planner",
    map_reduce_planner_to_tool_selection,  # type: ignore[arg-type, unused-ignore]
    ["tool_selection"],
)

def map_reduce_planner_to_tool_selection(state: OverallState) -> List[Send]:
    """Map each identified task in the planner stage to a tool_selection node."""
    return [
        Send(
            "tool_selection",
            {
                "question": task.question,
                "parent_task": task.parent_task,
            },
        )
        for task in state.get("tasks", list())
    ]

def tool_selection_output_router(state: ToolSelectionOutputState) -> Send:
    match state.get("next_action", ""):
        case "text2cypher":
            return Send("text2cypher", {"task": state.get("task", "")})
        case "predefined_cypher":
            return Send(
                "predefined_cypher",
                {
                    "task": state.get("task", ""),
                    "tool_call": state.get("tool_call", dict()),
                },
            )
        case "error":
            return Send("final_answer", dict())

main_graph_builder.add_edge("cypher_query", "summarize")
main_graph_builder.add_edge("predefined_cypher", "summarize")
main_graph_builder.add_edge("customer_tools", "summarize")
main_graph_builder.add_edge("summarize", "final_answer")

main_graph_builder.add_edge("final_answer", END)

When not to split into multiple agents

Most products do not need this. The same repository answers a support turn with a single graph that classifies the question into one of five paths (general, additional, graph, image, file) and runs it there (backend/app/lg_agent/lg_builder.py:94-96). That is one agent with five routes, and it is the right shape for a single turn.

The costs of splitting are concrete, and they are the reason to keep the task count small.

• Each task adds a model call and a merge

  Every Send runs tool selection on its own, and the merge is a separate summarisation node (backend/app/lg_agent/kg_sub_graph/agentic_rag_agents/workflows/multi_agent/multi_tool.py:128). Five tasks are five selections plus one summary, not one call.

• Retries are bounded on purpose

  The default is three attempts, and a query is not executed on the final attempt unless you opt in (backend/app/lg_agent/kg_sub_graph/agentic_rag_agents/workflows/multi_agent/multi_tool.py:65-66). That is a retry budget, not an agent loop; raising it trades latency for a chance at a valid query.

• A failed task narrows the answer silently

  An error task routes straight to final_answer (backend/app/lg_agent/kg_sub_graph/agentic_rag_agents/workflows/multi_agent/edges.py:87-90), so the summary is built without that task and nothing in the answer says so. The more tasks you fan out, the more ways the answer can quietly lose a part.

• Orchestration does not fix a missing data source

  In this build the knowledge-base ingestion path is not functional, and the README says so (README.md:19-20). Adding agents around a source that is not wired produces confident answers from fewer sources.

Split when the tasks use different tools, fail in different ways, and can be merged by a function. If any of the three is missing, one agent with a fixed tool list is the better system.