One mesh, every modality.
A request rarely lives in one modality. SutraMesh treats text, image, audio, documents, and structured data as a single coordinated pipeline — each handled by experts that share a common context.
What the mesh reasons over.
Text
Language understanding, summarization, structured extraction, and translation as first-class subtasks.
Image
Object recognition, scene understanding, OCR, and diagram parsing dispatched only when visual content is present.
Audio
Speech recognition, speaker context, intent, and acoustic understanding routed to dedicated audio experts.
Documents
Long-form comprehension over PDFs, contracts, and reports — combining layout, text, and retrieval.
Structured Data
SQL, JSON, tabular reasoning, and time-series analysis treated as their own modality, not text-with-extra-steps.
Future Modalities
Video, 3D, code execution traces, sensor streams. The mesh is designed to add modalities without redesigning the routing layer.
Across modalities, not just within them.
Most “multimodal” systems are really single models that accept multiple inputs. The model sees pixels and tokens together, but it answers in one forward pass and one voice. There is no structure to how the modalities interact.
Our research direction is different. We treat each modality as a first-class subgraph handled by experts that are good at it. Vision experts handle pixels. Document experts handle layout. Language experts handle the final synthesis. The router decides which subgraphs to spin up and how their outputs combine.
Three things follow from this:
- You only pay for modalities you use. A text-only request never triggers a vision expert.
- You can add new modalities without retraining the world. Adding video means adding video experts and a routing path — not a new foundation model.
- You get a trace, not a black box. The coordination across modalities is inspectable as a graph of expert calls.
Where multimodal coordination lives in the architecture.
The shared context layer is what makes cross-modal reasoning possible. The architecture page walks through it end-to-end.