Last week I posted about FlintAPI, a smart routing engine for AI models. Several people asked how the routing actually works under the hood. Here's the architecture.
The Pipeline: Decompose → Dispatch → Execute → Synthesize
When you send a prompt to model=flint, it doesn't just forward your request. It runs a four-stage pipeline:
1. Decompose
The decomposer (powered by a fast model like Qwen3.7-Max) analyzes your prompt and identifies which capability domains it needs:
"Compare Python and Go for API gateway"
→ domains: [research, analysis]
It outputs a structured decomposition: what sub-questions to ask, which domain each belongs to, and whether they can run in parallel.
2. Dispatch
Each sub-task is matched to the best model for that domain:
| Domain | Best Model | Why |
|---|---|---|
| Research | Qwen3.7-Max | Broad knowledge, strong recall |
| Analysis | DeepSeek V4 | Reasoning benchmarks |
| Code | DeepSeek V4 / Kimi K2 | Code generation scores |
| Content | GLM-5.2 | Writing quality |
The dispatcher also handles fallback: if a model call fails, it retries with the next-best model.
3. Execute
Independent sub-tasks run in parallel. Dependent ones (output of one feeds into the next) run sequentially. Each subtask gets context from the original prompt plus any upstream results.
4. Synthesize
Results from all sub-tasks are fed into a synthesizer model that:
- Cross-references facts between sub-results
- Flags contradictions
- Weaves everything into one coherent answer
- Adds a confidence score per source
The key decision: simple vs orchestrated
Not every prompt needs the full pipeline. The decomposer classifies complexity:
- Simple (single domain, factual) → single-model path, ~3-5 seconds
- Medium (2-3 domains) → parallel sub-tasks, ~15-40 seconds
- Complex (4+ domains, interdependencies) → full pipeline, ~60-90 seconds
This means "What's 2+2?" takes 2 seconds. "Compare three API gateways across five dimensions" takes 60 seconds and uses 4 models.
What I Learned Building This
Model selection is a real problem. For "write a Python function that uses dynamic programming", Qwen and DeepSeek give different approaches. Routing matters.
Synthesis is harder than decomposition. Getting 3 models to agree on facts and produce a unified answer is genuinely difficult. The synthesizer needs to detect and resolve contradictions.
SSE streaming saves user patience. 60 seconds feels like 5 when you see "Decomposing..." → "Running 3 sub-tasks..." → "Synthesizing..." in real time.
Try It
- Live demo (no signup): flintapi.ai/demo
-
API:
base_url="https://flintapi.ai/v1", model=flint - Code: github.com/moozechen/flintapi
What's the hardest AI routing problem you've encountered?
Quick survey — help me build the right thing:
What's the hardest AI routing problem you've run into? Drop it in the comments. I'll run your answer through Flint and share what its multi-model pipeline comes back with.
Also: how much time do you spend manually picking models? (a) never, (b) occasionally, (c) constantly — it's a pain
Top comments (0)