Multi-agent browser control

When we started building Scout's orchestration layer, the naive approach was a single AI agent that handled everything: navigation, interaction, network monitoring, storage management. One model. One context window. One point of failure.

It worked — right up until the tasks got complex. A workflow that needed to intercept network responses while simultaneously filling forms while emulating a mobile device would hit context limits, lose track of its place, and produce unpredictable results. We needed a better model.

Scout's multi-agent graph

Scout's orchestration layer runs a multi-agent graph. The graph has one Orchestrator agent at the root and six specialist sub-agents: Interactor, Network Manager, Storage Manager, Media Controller, Debugger, and Emulator. Each sub-agent owns a specific category of tools.

The Orchestrator handles session management, navigation, and content extraction directly. For everything else, it delegates to the appropriate specialist. The specialist runs its tools, returns structured results, and the Orchestrator integrates them into the overall task.

// Scout agent graph bootstrap (simplified)
const orchestrator = new Agent({
  name: "orchestrator",
  tools: [
    browserTabs,
    browserConnect,
    browserNavigate,
    browserSnapshot,
    browserExtract,
  ],
});

const interactor = new Agent({
  name: "interactor",
  tools: [
    browserAction,
    browserSelect,
    browserAttach,
    browserDialog,
    browserScroll,
  ],
});

const networkManager = new Agent({
  name: "network-manager",
  tools: [
    browserNetwork,
    browserRoute,
    browserUnroute,
    browserHar,
    browserWebsocket,
  ],
});

export const scoutGraph = createGraph({
  agents: { orchestrator, interactor, networkManager /* ... */ },
  entrypoint: "orchestrator",
});

Why specialization matters

Each specialist has a focused tool set and a focused system prompt. The network manager knows everything about request interception, HAR recording, and WebSocket monitoring — and nothing about CSS selectors or form fields. This focus makes each agent more reliable and its outputs more predictable.

It also makes the system composable. We can swap out the LLM backing the emulator sub-agent without touching the orchestrator. We can add new capabilities by adding a new specialist without expanding the orchestrator's already-busy context window.

Dynamic model routing

Different tasks have different latency and cost profiles. The orchestrator's "navigate and understand" loop runs constantly and needs to be fast. A "debug why this test is flaking" task is rare and benefits from a more capable model. Scout's dynamic model selection lets us route different categories of tasks to different providers at runtime based on configuration.

In practice: session, navigation, and content tools use a fast Flash-class model. Network analysis and debugging tasks route to a more capable model when the agent decides the complexity warrants it.