MCP server architecture: one server per module, one process per session

How MCP servers are usually wired

The common shape is a single process that declares its tools in code and speaks stdio, with the client launching it from a command and passing configuration through arguments and environment variables. That works well for one server: the tool list is code, the transport is the client's problem, and credentials arrive as env values.

The same shape extends to remote deployments through an HTTP transport, where one process serves many callers and authentication happens per request. Registry or config files then map a server name to a command, its arguments and its environment.

Both are reasonable defaults. They get awkward in two specific places: a long-lived stdio process cannot hold several users' tokens at once, and a single server that fronts every backend tends to accumulate the retrieval logic of all of them.

One MCP server per module, described by three fields

The gateway in this repository deliberately does not implement retrieval. Its own header states the layer exists to orchestrate MCP connections, and that every module is expected to ship its own MCP server.

What the gateway keeps instead is a descriptor per module — a command to launch, and the name of the environment variable that will carry the caller token. Nothing else crosses that boundary, so a module can change its internal retrieval without the gateway learning about it.

Registration is opt-in: a module appears in the descriptor list only when it declares both the command and the token variable name. A module that declares neither is simply not exposed. Asking for a descriptor of a module that has no server throws instead of returning an empty shape, so a missing server fails at session start rather than producing an agent with no tools.

export type ModuleMcpDescriptor = {
  moduleId: ModuleId;
  command: string;
  envTokenName: string;
};

export function getModuleMcpDescriptors(): ModuleMcpDescriptor[] {
  return getRegisteredModules()
    .filter((module): module is typeof module & { mcpCommand: string; mcpEnvTokenName: string } =>
      Boolean(module.mcpCommand && module.mcpEnvTokenName),
    )
    .map((module) => ({
      moduleId: module.id,
      command: module.mcpCommand,
      envTokenName: module.mcpEnvTokenName,
    }));
}

The server side: three tools, no prose

The GraphRAG module ships a FastMCP server whose tool surface is three functions: search, list sources, get document. Each returns data for the agent to answer from; the tools do not compose an answer themselves.

Two details are worth copying. List-sources creates a private source for the caller when one does not exist yet and the caller is not an admin, so the tool is safe to call on a fresh account. And every tool resolves the user context from the environment on each call, which is what makes the per-session process model work.

mcp = FastMCP("graph-rag")

@mcp.tool()
async def graph_search(query: str, limit: int = 10, source_id: str | None = None, mode: str = "mix") -> dict:
    """Search GraphRAG and return evidence/context for the Agent to answer from."""
    return await search(resolve_user_context(), query, limit, source_id=source_id, mode=mode)

Credentials: a process per session, a token from the environment

stdio is a process-scoped channel, so one process cannot honestly serve several users. The launcher therefore builds a launch plan per agent session: one subprocess per active module, with the caller bearer token injected into the variable the module named in its descriptor, alongside the user id, name and admin flag.

The server fails closed. If the token variable is absent it raises instead of starting in an anonymous mode, which is the behaviour you want when the process is spawned per session: a missing token is a wiring bug, not a degraded state.

On the client side the tools are allowlisted per module, and the allowlist is the same three names the server registers. Tools that are not on the list never reach the agent, and one high-risk tool is withheld by default until a confirmation step exists.

def resolve_user_context() -> UserContext:
    # Stdio MCP is process scoped. Agent Gateway should launch one module MCP
    # process per user session and inject the user's bearer token through TOKEN_ENV.
    token = os.getenv(TOKEN_ENV)
    if not token:
        raise RuntimeError(f"{TOKEN_ENV} is not configured")
    return UserContext(
        user_id=os.getenv("FF_USER_ID", ""),
        username=os.getenv("FF_USERNAME", ""),
        is_admin=os.getenv("FF_IS_ADMIN") == "true",
    )

export function buildPerUserMcpLaunchPlan(input: AgentSessionInput): McpSessionLaunchPlan {
  const descriptor = getModuleMcpDescriptor(input.activeModule);
  return {
    moduleId: input.activeModule,
    command: descriptor.command,
    env: {
      [descriptor.envTokenName]: input.bearerToken,
      FF_USER_ID: input.user.userId,
      FF_USERNAME: input.user.username,
      FF_IS_ADMIN: String(input.user.isAdmin),
    },
  };
}

const GRAPH_RAG_TOOLS = new Set([
  'graph_search',
  'graph_list_sources',
  'graph_get_document',
]);

When this shape is wrong

This design trades startup cost for credential isolation. It is a poor fit when you do not need the isolation:

• One backend, one operator, no per-user data. Then the registry is indirection with no payoff, and a single stdio entry in a client config is simpler.
• Stateless shared tools. A subprocess per session pays a process start on every session and cannot share a cache or a connection pool; a long-lived server with per-request auth is cheaper.
• Remote multi-tenant serving. The repository itself notes that Streamable HTTP with a per-request signed token is the replacement path once a module supports it; until then, per-session subprocesses are a local-first answer.
• Anonymous or trial access. The fail-closed check means the server will not run without a token. If you need an unauthenticated mode, this is not the starting point.
• Large tool sets per module. Every tool is a name the agent can pick; three focused tools are easier to route than thirty, and an allowlist only helps if it stays short enough to read.