A small model that decides how to think.
The router is the smallest, fastest model in the mesh — and the most important. It decides what every request actually becomes: which experts run, in what order, and when to stop.
Routing is the central problem.
In most AI products today, “routing” is glue: a regex, an intent classifier, or a hand-written switch on top of one big model. The model still does almost all the work, and the system still pays the cost of that model on every request.
We think this is backwards. In a mesh of specialists, the router is the system. It is the only component that sees every request, that knows what experts exist, and that decides what intelligence to spend. The cost, latency, and capability profile of the whole mesh is downstream of how good the router is.
That makes the router a serious modelling problem in its own right — not a piece of plumbing.
What we think a good router looks like.
Task decomposition
The router parses intent, identifies the modalities involved, and produces a typed subtask graph — not just a single label.
Expert selection
Routing is a calibrated classification problem: pick the expert with the best expected utility under capability, cost, and latency constraints.
Lightweight inference
The router must be orders of magnitude cheaper than the experts it dispatches to. A heavy router defeats the purpose of orchestration.
Calibrated confidence
Knowing when not to route — and falling back to a stronger generalist — is as important as routing correctly when confident.
Observability
Every routing decision is a first-class trace: which experts were considered, which scored highest, and why a path was chosen.
Cost-aware
Routing optimizes a real objective: quality per dollar, per millisecond. The mesh prefers cheap correct paths to expensive plausible ones.
What we are studying.
- How small can the router be? What is the capability floor below which routing accuracy collapses for real-world, multi-turn, multimodal inputs?
- How is the router trained? Supervised data, RL from execution traces, distillation from a stronger planner, or all three combined.
- How does it know its experts? Static capability descriptors, learned embeddings, or empirical performance estimates updated online.
- How does it handle uncertainty? Abstention, escalation to a more capable router, or asking the user — and when each is appropriate.
- How does it stay honest? Avoiding overconfident routing on out-of-distribution inputs is harder than maximising average accuracy.
See where the router fits in the mesh.
The router is one stage of a larger execution model. Read the full architecture for the surrounding pipeline.