DEV Community

Code Nomi Nomi
Code Nomi Nomi

Posted on

Why 86% of an AI agent's decisions shouldn't touch an LLM

#ai #architecture

Breaking down our local architecture routing 86% deterministic code, 11% SQLite, and 3% LLM on an i7-7700 with 0€ monthly API costs.

https://alice.matourdecontrole.fr

In an autonomous agent system, most decisions are not speculative[cite: 10]:

  • Is this action permitted[cite: 10]?
  • Does this database record exist[cite: 10]?
  • Is this field mapped[cite: 10]?
  • Did the unit test pass[cite: 10]?

Defaulting these checks to a remote language model introduces network latency, recurring API costs, and probabilistic drift[cite: 10].

In our architecture (Alice), we inverted the execution pipeline[cite: 10]: deterministic logic resolves first, local memory follows, and the language model is called strictly as a fallback[cite: 10].

The Request Breakdown

Here is the empirical split measured across live workloads[cite: 10]:

  • 86% — Handled directly by code (0.23 s · 0 € · 0 LLM calls): Deterministic gates, regex, and state machine transitions resolve the vast majority of operations instantly[cite: 10].
  • 11% — Resolved via local memory (0.65 s · local SQLite): Retrieved from an indexed store of 9,153 passages[cite: 10].
  • 3% — The LLM layer: Reserved exclusively for open-ended or novel inputs that local rules cannot arbitrate[cite: 10].

Running at 0 € / Month on 2017 Hardware

The entire system operates on a commodity host (Intel Core i7-7700, 8 GB RAM)[cite: 10]:

  • No external API dependency: 0 € monthly billing[cite: 10].
  • Sovereign storage: LUKS-encrypted, zero bytes leave the machine[cite: 10].
  • Reliability: 56 TDD unit tests passing green[cite: 10].

Two deterministic guards sit in front of the model layer[cite: 10]:

  1. A guard that explicitly refuses to process queries outside its verified context[cite: 10].
  2. A math/execution router that delegates calculation directly to runtime logic rather than token generation[cite: 10].

If the network drops, the system degrades gracefully: the remaining 3% slows down, but the core 86% continues to execute without interruption[cite: 10].

Question

When building production agents on sensitive or private data, how do you segment deterministic code vs. model inference? Do you handle routing through an FSM, or do you still pass control flow to the prompt layer?

Top comments (0)