Project 4 of Agentic AI from Zero is a multi-tool orchestrator, hand-rolled with no framework, and I ran it for real against NVIDIA NIM's meta/llama-3.1-8b-instruct — every routing and synthesis call a live HTTP/1.1 200 OK to integrate.api.nvidia.com. One run exercises five ideas: a dynamic tool registry, capability-based routing, deny-by-default permissions, parallel execution, and conflict resolution. Everything below is verbatim from that run.
The tools register themselves
Six tools self-register via an @tool(...) decorator the moment their module is imported — each declaring a name, a capabilities set, a permission scope, and a JSON arg schema. The router never sees tool names; it only ever queries the registry by capability. Add a new tool and it becomes routable without touching the router — the registry is the only thing that changes.
The six tools span the spectrum on purpose: three price feeds (price_alpha/beta/gamma, all network), a read-only weather_lookup and fx_convert, and one ledger_write that is the sole write tool. Three feeds — not two — so a majority vote is actually meaningful, and each carries a trust priority and an as_of timestamp so the tie-breaks are fully specified.
Routing on capabilities, not names
Asked "what is the current reference price of AAPL?", the model is shown the advertised capabilities (price_quote, weather, persist, …), not the tools, and it picks the capabilities the task needs. The orchestrator resolves those to concrete tools. In the run: route via=llm caps=['price_quote'] → ['price_alpha', 'price_beta', 'price_gamma']. Routing is validated against the registry — hallucinated capabilities are dropped — with a keyword fallback logged as via=fallback so a small 8B model can never wedge the pipeline.
Deny-by-default permissions — a real refusal
I ran the same "record an audit note in the ledger" task twice. Under a restricted grant of ['network','read'], routing lands on ledger_write, which needs write — so it is DENIED and logged, and the run answers that it couldn't. Grant ['network','read','write'] and the identical routing now writes the ledger entry. The only difference was the scope. The enforcement is pure Python — the safety decision never depends on the model.
Parallel execution — 2.97× measured
The three price feeds are independent, so they run concurrently in a thread pool. Each feed cost 0.60s: run serially that's 1.80s wall-clock, run concurrently it's 0.61s — a measured 2.97× faster over the exact same tool set. It's a real A/B timed in the same run, not a claim in a README — the same three feeds, once one after another and once at the same time, with the saving printed.
Conflict resolution — the feeds disagreed
The three feeds didn't agree: alpha $150.25, beta $172.40, gamma $150.25. A documented, deterministic policy settles it — majority first, then trust-priority, then freshness — so $150.25 wins on a 2-to-1 vote, and the conflict is recorded: all three prices, who disagreed, and which policy step decided. The deliberate twist: beta is actually the most-trusted feed by priority, yet majority-first correctly outvotes it. Swap the policy order and the winner changes — which is exactly why the policy is written down rather than left implicit.
What's stable across re-runs
An 8B model isn't perfectly deterministic even at temperature 0, so a re-run may phrase the routing reason or final answer differently. But the capabilities it selects, the denial, the timing shape, and the conflict winner are stable — because routing is validated against the registry, and permission enforcement and conflict resolution are deterministic Python, not model output. Only the routing and the final synthesis are the LLM.
The full recorded run:
Live https://dev48v.infy.uk/agentic/project4-orchestrator.html
Repo https://github.com/dev48v/agentic-ai-from-zero
Top comments (0)