COMPREHENSIVE ANALYSIS: THE CALIBRATION PHASE (EXP01–07) I built an epistemic gate to stop LLM data poisoning during fine-tuning.

Eduardo — Mon, 07 Sep 2026 07:38:22 +0000

📅 First, the complete timeline
Date EXP
Aug 19 01, 02, 03 Genesis: symbolic filter, statistical rigor, first collapse
Aug 20 04 The Manual’s five policies; A→Gate→B is born
Aug 22 05 “Beatriz” is named; the Softplus formula is born; first GPU
Aug 25 06 Free-form text; keywords; name + PolyForm license
Aug 26 07→08→09→10→11 Vector gate → π_ref → LoRA → Qwen → TinyLlama (five in one day!)
Aug 27 12 Cache + held-out set; invisibility measured
Aug 28 13, 14 Phi-3 and Pythia
Sep 1–4 15, 16 Ablation and final validation

The entire project—from the first line of code to OpenTimestamps—took 17 days. And all of it was orchestrated on a 2006 Toshiba.

🧬 The four parallel evolutions that calibration resolved

The loss function: from runaway punishment to bounded contrast

This is THE story of calibration—the “collapse” mentioned above:
EXP Mechanism Result
01–02 Filtering only (no penalty) Works, but is passive
03 Raw −logP(rejected) 🔥 THE COLLAPSE: loss diverges to −27, margin reaches +13, but the model destroys itself—the unbounded negative gradient is a bomb
04 −logP(argmax), brought under control It no longer explodes (loss ~−0.6)... but it is almost useless (+0.48). Meanwhile, REWRITE dominates (+10.9)
05 CE(truth) + β•softplus(m + logP(false) − logP(truth)) ✨ The synthesis: relative, bounded punishment. It never changed again

The distilled lesson: do not punish falsehood in absolute terms—it diverges. Punish it only relative to the truth, using a function that switches itself off (Softplus). That is exactly what a reviewer would ask: “Why Softplus instead of a simple negative gradient?” You have the empirical answer in EXP03 vs. EXP05.

The gate: from dictionary lookup to geometry

EXP01–04: Exact lookup (Pydantic + dict)—perfect, but only for triplets.
EXP06: Word overlap—the first attempt with free-form text, but fragile (erratic margins: 1.38→4.23→0.82).
EXP07: Embeddings + cosine similarity (DenseVectorGate)—and the margins suddenly stabilize: +10.2/+10.1/+10.6, nearly identical across seeds.

The EXP06 vs. EXP07 comparison provides the empirical justification for the vector gate: the same conceptual corpus and the same protocol, yet the shift from lexical overlap to semantic geometry multiplies the margin by approximately 15× and eliminates noise. Another finding that remains implicit today.

The corpus: from toy examples to the real world

EXP01–04: entity_a color = blue—abstract and free from pretraining contamination.
EXP06: Six real-world facts in free-form text.
EXP07: The final eight facts + keywords + UNKNOWN_POOL—the exact format carried through to EXP16.

Starting with synthetic entities was methodologically sound: it isolated the filter’s mechanics from the model’s priors. Only after the mechanism had been proven did the project move on to real-world facts.

Rigor: present from the very beginning

EXP01 was already verifying the SHA-256 hashes of the input weights. EXP02 already included seeds [11,22,33] and a PLACEBO branch. Reproducibility was not added later—it was part of the project’s DNA from day one.

(Fun fact: z3-solver was installed in EXP01–04 but never invoked.)

💎 THE MOST BEAUTIFUL NARRATIVE DISCOVERY IN THE ENTIRE SERIES

Look at this. In EXP04, the HYBRID branch is defined as REWRITE + EPISTEMIC... but because of the bug I detected, it ended up being identical to EPISTEMIC—the REWRITE component was never executed. The branch “failed.”

But what is Beatriz, from EXP05 onward?

When the gate detects CONTRADICTED, it trains on the true anchor = REWRITE
It also applies the Softplus contrast against the falsehood = a controlled EPISTEMIC

Beatriz IS the HYBRID that the bug prevented us from testing in EXP04. The branch that failed because of an implementation error became the definitive architecture three days later—implemented correctly and with a bounded penalty.

The bug did not kill the idea. The idea was right, and it was reborn with better mathematics. That is a story of real science worth telling. 🐛→🦋

🔍 EXP07 vs. EXP08: the 55 minutes that proved something

I compared the runs using the same seed and the same stream:
EXP07 (without π_ref) EXP08 (with π_ref)
BEATRIZ margin +10.2 / +10.1 / +10.6 +10.6 / +11.0 / +11.0
BEATRIZ PPL 948 / 1145 / 1140 1098 / 1121 / 2082

Conclusion: π_ref anchoring (EXP08) did not solve the perplexity explosion—the PPL values were comparable or even worse. What solved it was LoRA (EXP09): PPL 43–91.

In other words, the effective constraint was not the reference-based loss, but the low-rank architecture. This explains why π_ref was abandoned after EXP08. It is also an implicit mini-ablation—EXP07 vs. EXP08 vs. EXP09—that deserves a paragraph in the series:

“We tested the DPO-style solution and the architectural solution—and the architectural one won.”

NOTE: “16 experiments (EXP08–EXP16)” → “16 experiments: EXP01–07 focused on anti-collapse calibration, and EXP08–16 on definitive results.”

I will soon continue publishing the content and results from EXP08–EXP16. For now, I have only uploaded EXP08 and EXP09 to my repository. Here, I have shown you the process, so stay tuned for the upcoming experiments.

Sincerely,
Eduardo Ayala, creator of Beatriz

Demonstrating that true innovation in AI is born from freedom, rigor, and open-source code.

https://github.com/EduardoAyalaT/beatriz-epistemic-gate-

I built an epistemic gate to stop LLM data poisoning during fine-tuning. Tested across 5 architectures, orchestrated on a 2006 Toshiba laptop for $0.

Eduardo — Fri, 04 Sep 2026 23:41:10 +0000

Hi everyone,
For a long time, the AI industry has pushed the narrative that advanced safety research, data poisoning auditing, and model alignment require massive clusters and millions of dollars.
I wanted to test if that's true. Over the last few months, using a 2006 Toshiba Satellite (2GB RAM) for orchestration and free Kaggle T4 GPUs ($0 cost), I built and ran a 16-experiment series (EXP08–EXP16) exploring epistemic collapse and data poisoning in LLMs.
Today, I'm open-sourcing the first phase of this work: Beatriz Epistemic Gate.
What problem does it solve?
Malicious data poisoning during fine-tuning is often surgical and invisible to aggregate metrics (like standard perplexity). An attacker can destroy specific facts down to an exact indifference point while global metrics actually improve because of generic fine-tuning on fluent prose. This makes silent poisoning a stealthy threat for small teams and startups fine-tuning local models.
How does Beatriz work?
Instead of invading or rewriting complex training loops, Beatriz acts as a lightweight, non-invasive defensive proxy (~0.1ms decision latency) that sits between a generative source and a training student model. It verifies generated text against an immutable anchor corpus using cosine similarity in embedding spaces, applying a composite loss function:
• LceLce: Preserves linguistic fluency.
• LcontrastivaLcontrastiva (Softplus): Keeps active pressure on the model to anchor to truth even after cross-entropy saturates.
Key Results (EXP08–EXP16):
• Multi-architecture validation: Tested across GPT-2 (124M), Qwen-2.5-0.5B, TinyLlama-1.1B, Pythia-1.4B, and Phi-3-mini-4k-instruct (3.8B).
• Ablation (EXP15): Pure filtering (GATE_ONLY) provides 65% of the defensive benefit out of the box. The contrastive term adds the remaining 35%.
• Scaled Held-Out (EXP16): Validated on a rigorous 30 multi-domain unattacked facts benchmark, consolidating a robust truth margin of +4.19 ± 0.08 with 0.93 Precision and 0.80 Recall on the gate.
What's in the repo right now?
To avoid overwhelming the community, I'm releasing the project in phases. Today you'll find:

EXP08 & EXP09 code & notebooks (fully reproducible on Kaggle).
Technical Whitepaper (WHITEPAPER.md / WHITEPAPER.es.md) detailing all 16 experiments.
Corrective Manual (docs/MANUAL_CORRECTIVO.md) with formal mathematical loss specifications. If you're an indie dev, researcher, or startup exploring safe local fine-tuning, check it out, run the notebooks, and let me know your thoughts! GitHub Repo: [https://github.com/EduardoAyalaT/beatriz-epistemic-gate-]

DEV Community: Eduardo

COMPREHENSIVE ANALYSIS: THE CALIBRATION PHASE (EXP01–07) I built an epistemic gate to stop LLM data poisoning during fine-tuning.

I built an epistemic gate to stop LLM data poisoning during fine-tuning. Tested across 5 architectures, orchestrated on a 2006 Toshiba laptop for $0.