Six months ago I set out to make LLMs "smarter" by orchestrating many of them
together. I measured it. It didn't work. Here's what I shipped instead — and why
the failure is the part I'm proudest of.
The idea that failed (and why I'm telling you)
The plan was "mitosis": split a task across several LLMs, let them multiply and
compete, then synthesize the best answer. It sounds great in a pitch deck.
On ground-truth executed tests, it made correctness worse:
- Baseline 95% → mitosis 83% (−17 passing tests; synthesis corrupted answers that were already correct)
- At 4–6× the cost
- Confirmed across three independent experiments (code correctness, security remediation, verified cross-model selection)
Every gain was ≤ 0. So I deleted it. The full evaluation — including the failure —
is in the repo's FINDINGS.md.
The lesson: an idea that survives a pitch is not the same as an idea that
survives a measurement.
What actually worked — and shipped
BIOMA is a small, local, provider-agnostic kernel (Rust core + a thin Python
layer) that sits in front of any LLM call and hardens the payload in-process,
before it leaves your machine.
from bioma.firewall_client import CognitiveFirewall
fw = CognitiveFirewall(vault={"db_password": DB_PW}) # secrets to protect
h = fw.shield(history, "refactor this function")
# h.prompt / h.system -> clean, dehydrated, secret-free payload
# h.telemetry -> saturation, red_alert, apoptosis_reduction, kernel_latency_us
import anthropic # or google.genai, or openai
msg = anthropic.Anthropic().messages.create(
model="claude-sonnet-5", max_tokens=1024,
system=h.system or "", messages=[{"role": "user", "content": h.prompt}])
Three mechanisms, all measured:
1. Efficiency — context apoptosis
Each context block gets a metabolic weight and a half-life; low-value blocks (old
logs, resolved chatter) are purged before dispatch.
- −80% input tokens typically; up to −97% on long, noisy sessions.
- A real 16-round session: 47,890 → 2,022 input tokens, apoptosis latency ~1.6µs, 0/16 dispatch errors.
2. Security — a cognitive firewall
- Secret redaction: vaulted values never reach the model (inbound and in the response).
-
Cognitive-DDoS / prompt-flood detection: an n-gram saturation scan flags
floods →
0x0Fred alert → apoptosis. - Timeout guard on every dispatch.
- Red-team run: 0 secrets leaked (2 redacted); a 32,317-token flood dehydrated to 13 tokens in 0.6µs; a code-injection loop contained by timeout.
3. Speed — a lock-free hormonal bus
An atomic in-memory signalling substrate (~5µs) carries the alert state.
(Throughput benched at ~2M signals/s.)
No lock-in
Anthropic, Google, OpenAI, or a local model — same layer. You harden the payload
here and hand it to your SDK.
Why fair-source, not open source
The license is FSL-1.1-MIT: the code is source-available (read it, run it,
build on it), free for any non-competing use, and it auto-converts to MIT
after two years. I'm a solo dev — I wanted it visible and auditable without
someone reselling it as-is. It's not OSI open source, and I'd rather say that
plainly than blur the line.
The point
BIOMA isn't magic. The whole thing is one discipline: measure everything, and
keep only what survives the measurement — even when that means deleting the
feature you started with.
Repo (Rust + Python, benchmarks, and the honest FINDINGS.md):
https://github.com/jonathascordeiro20/bioma-framework
What would you attack first? I'll be in the comments — especially happy to go deep
on the firewall's saturation heuristic or the mitosis eval.
Top comments (5)
Deleting your own best idea is often the real engineering move.
LLM efficiency and security work has a temptation to become clever very quickly. The better question is usually: what is the smallest control that makes the system easier to reason about under load, failure, and weird model behavior?
If an idea improves the demo but makes the operational story harder to trust, cutting it is probably a feature.
Exactly. That became the biggest lesson from this project. The multi-agent "mitosis" idea looked impressive conceptually, but measurements showed it reduced correctness while increasing cost. Removing it made the system simpler, more predictable, and easier to trust. For this project, deleting code was as important as writing it.
That is the cleanest lesson in the project. The impressive idea was not the useful idea.
I like that you measured the extra agent layer against correctness and cost instead of keeping it because it made the architecture sound more advanced. That is a hard habit to build because the removed code rarely gets credit, even when it is the thing that made the system more trustworthy.
Thank you so much for capturing the exact essence of this takeaway!
It’s incredibly tempting to keep a complex architecture just because it "sounds elegant" or makes the pitch deck look smarter[cite: 1]. But at the end of the day, every unnecessary line of code we keep is a technical liability we’ll end up paying for in maintenance and latency.
Deleting "mitosis" definitely bruised my ego on day one[cite: 1], but once I saw the tests running with higher reliability and at a fraction of the cost, it became clear: the best code is either the one you never have to write, or the one you have the courage to delete.
Appreciate the feedback!
That is the painful but useful test: if the elegant layer cannot beat the simple path on reliability, cost, and maintenance, it is probably serving the pitch more than the product. Deleting it is not a retreat. It is the moment the architecture starts answering to the benchmark instead of to the diagram.