Six months ago I set out to make LLMs "smarter" by orchestrating many of them
together. I measured it. It didn't work. Here's what I shipped instead — and why
the failure is the part I'm proudest of.
The idea that failed (and why I'm telling you)
The plan was "mitosis": split a task across several LLMs, let them multiply and
compete, then synthesize the best answer. It sounds great in a pitch deck.
On ground-truth executed tests, it made correctness worse:
- Baseline 95% → mitosis 83% (−17 passing tests; synthesis corrupted answers that were already correct)
- At 4–6× the cost
- Confirmed across three independent experiments (code correctness, security remediation, verified cross-model selection)
Every gain was ≤ 0. So I deleted it. The full evaluation — including the failure —
is in the repo's FINDINGS.md.
The lesson: an idea that survives a pitch is not the same as an idea that
survives a measurement.
What actually worked — and shipped
BIOMA is a small, local, provider-agnostic kernel (Rust core + a thin Python
layer) that sits in front of any LLM call and hardens the payload in-process,
before it leaves your machine.
from bioma.firewall_client import CognitiveFirewall
fw = CognitiveFirewall(vault={"db_password": DB_PW}) # secrets to protect
h = fw.shield(history, "refactor this function")
# h.prompt / h.system -> clean, dehydrated, secret-free payload
# h.telemetry -> saturation, red_alert, apoptosis_reduction, kernel_latency_us
import anthropic # or google.genai, or openai
msg = anthropic.Anthropic().messages.create(
model="claude-sonnet-5", max_tokens=1024,
system=h.system or "", messages=[{"role": "user", "content": h.prompt}])
Three mechanisms, all measured:
1. Efficiency — context apoptosis
Each context block gets a metabolic weight and a half-life; low-value blocks (old
logs, resolved chatter) are purged before dispatch.
- −80% input tokens typically; up to −97% on long, noisy sessions.
- A real 16-round session: 47,890 → 2,022 input tokens, apoptosis latency ~1.6µs, 0/16 dispatch errors.
2. Security — a cognitive firewall
- Secret redaction: vaulted values never reach the model (inbound and in the response).
-
Cognitive-DDoS / prompt-flood detection: an n-gram saturation scan flags
floods →
0x0Fred alert → apoptosis. - Timeout guard on every dispatch.
- Red-team run: 0 secrets leaked (2 redacted); a 32,317-token flood dehydrated to 13 tokens in 0.6µs; a code-injection loop contained by timeout.
3. Speed — a lock-free hormonal bus
An atomic in-memory signalling substrate (~5µs) carries the alert state.
(Throughput benched at ~2M signals/s.)
No lock-in
Anthropic, Google, OpenAI, or a local model — same layer. You harden the payload
here and hand it to your SDK.
Why fair-source, not open source
The license is FSL-1.1-MIT: the code is source-available (read it, run it,
build on it), free for any non-competing use, and it auto-converts to MIT
after two years. I'm a solo dev — I wanted it visible and auditable without
someone reselling it as-is. It's not OSI open source, and I'd rather say that
plainly than blur the line.
The point
BIOMA isn't magic. The whole thing is one discipline: measure everything, and
keep only what survives the measurement — even when that means deleting the
feature you started with.
Repo (Rust + Python, benchmarks, and the honest FINDINGS.md):
https://github.com/jonathascordeiro20/bioma-framework
What would you attack first? I'll be in the comments — especially happy to go deep
on the firewall's saturation heuristic or the mitosis eval.
Top comments (1)
Deleting your own best idea is often the real engineering move.
LLM efficiency and security work has a temptation to become clever very quickly. The better question is usually: what is the smallest control that makes the system easier to reason about under load, failure, and weird model behavior?
If an idea improves the demo but makes the operational story harder to trust, cutting it is probably a feature.