DEV Community: Eber Cruz Fararoni

The End of Destructive AI Hallucinations: Hybrid Kernel Architecture with Java 25 and Zero-Trust Guardrails

Eber Cruz Fararoni — Wed, 15 Apr 2026 13:32:40 +0000

Integrating Deterministic Routing and Zero-Trust Guardrails

Eber Cruz Fararoni — Software Engineer | C-FARARONI Project
March 2026 · Architecture Notes
Fararoni Kernel: Deterministic Routing + Zero-Trust Protection Triad

Abstract

The adoption of Large Language Models (LLMs) in software development faces two critical barriers: the stochastic nature that causes destructive hallucinations and the high latency on trivial commands. This article presents the Fararoni Kernel, a hybrid execution architecture that solves both problems simultaneously through two complementary contributions:

(1) Deterministic Routing (Levels 1-2): A 5-level execution cascade that intercepts system commands (pwd, ls, git) and maps natural intentions ("do the commit") directly to shell sequences, removing the LLM from tasks where its intervention introduces unnecessary latency and hallucination risk.

(2) Zero-Trust Guardrails (Protection Triad): A defense-in-depth mechanism that acts on the stochastic levels (3-5), implementing a Kill-Switch based on Jaccard/Volume, transactional isolation via ephemeral Git branches (Saga Pattern), and atomic recovery via Shadow Backups.

Empirical results demonstrate a 90% latency reduction for operational commands, a 99.99% destructive hallucination blocking rate, and 0% permanent data loss.

Keywords: Hybrid Kernel, Deterministic Routing, LLM Hallucinations, Zero-Trust Architecture, Defense in Depth, AI-Assisted Software Engineering.

1. Introduction

1.1 Problem Statement

The integration of LLM agents in development workflows presents a fundamental dilemma: the same model that can refactor complex code also hallucinates responses for a simple pwd, inventing paths like /home/user when the actual directory is /Users/the/Projects/microservice. Worse, when a 7B parameter LLM receives 31 tools and is asked "do the commit", it can take 10 seconds and respond with tool call JSONs printed as plain text instead of executing them.

These problems reveal an architectural flaw: treating every input as a task requiring LLM inference is inefficient and insecure. Deterministic commands (pwd, git status, ls) should not go through a probabilistic inference process. Clear intentions ("do the commit") should not depend on a 7B parameter model's reasoning capabilities.

1.2 Limitations of Current Solutions

Tool	Approach	Limitation
Pure Agent Architectures	Everything through the LLM	Unnecessary latency on simple commands
NeMo Guardrails	Content filter	Does not protect structural code integrity
Prompt Engineering	Model instructions	20% leakage rate in production
IDEs with AI	Human validation	Not scalable in autonomous workflows

No current solution simultaneously addresses the latency problem (when to wake the LLM) and the integrity problem (how to protect code when the LLM operates).

1.3 Contribution

We propose an inversion of control in two dimensions:

Routing: Instead of sending everything to the LLM, the Kernel decides the minimum complexity level needed to resolve each input.
Protection: Instead of making the model perfect, we build a deterministic environment that makes permanent data destruction impossible.

2. Hybrid Kernel Architecture

2.1 General Vision: 5-Level Cascade

                    ┌─────────────────────────┐
                    │      USER INPUT         │
                    └────────────┬────────────┘
                                 │
    ╔════════════════════════════╪═══════════════════════════╗
    ║  DETERMINISTIC ZONE (No LLM)                           ║
    ║                            │                           ║
    ║  ┌─────────────────────────┴─────────────────────────┐ ║
    ║  │ LEVEL 1: BARE COMMANDS                            │ ║
    ║  │ pwd, ls, git status                    ~0ms Shell │ ║
    ║  └─────────────────────────┬─────────────────────────┘ ║
    ║                       null?│                           ║
    ║  ┌─────────────────────────┴─────────────────────────┐ ║
    ║  │ LEVEL 1.5: COMPOSITE                              │ ║
    ║  │ "do the commit"                 ~0ms Macro + Shell│ ║
    ║  └─────────────────────────┬─────────────────────────┘ ║
    ║                       null?│                           ║
    ║  ┌─────────────────────────┴─────────────────────────┐ ║
    ║  │ LEVEL 2: GGUF LOCAL                               │ ║
    ║  │ "hello", "thanks"               ~1s Light LLM     │ ║
    ║  └─────────────────────────┬─────────────────────────┘ ║
    ╚════════════════════════════╪═══════════════════════════╝
    ─────── DETERMINISTIC / STOCHASTIC FRONTIER ───────────────
    ╔════════════════════════════╪═══════════════════════════╗
    ║  STOCHASTIC ZONE (With LLM)                            ║
    ║                            │                           ║
    ║          ┌─────────────────┴────────────────┐          ║
    ║          │   PROTECTION TRIAD               │          ║
    ║          │   ← Active here                  │          ║
    ║          └─────────────────┬────────────────┘          ║
    ║                            │                           ║
    ║  ┌─────────────────────────┴─────────────────────────┐ ║
    ║  │ LEVEL 3: TOOL CALLING                             │ ║
    ║  │ 31 tools                        8-10s LLM + Tools │ ║
    ║  └─────────────────────────┬─────────────────────────┘ ║
    ║                       null?│                           ║
    ║  ┌─────────────────────────┴─────────────────────────┐ ║
    ║  │ LEVEL 4: THINKING                                 │ ║
    ║  │ Deep reasoning               10-15s DeepSeek/Qwen3│ ║
    ║  └─────────────────────────┬─────────────────────────┘ ║
    ║                       null?│                           ║
    ║  ┌─────────────────────────┴─────────────────────────┐ ║
    ║  │ LEVEL 5: FALLBACK                                 │ ║
    ║  │ Plain VllmClient                2-5s No tools     │ ║
    ║  └───────────────────────────────────────────────────┘ ║
    ╚════════════════════════════════════════════════════════╝

2.2 Key Principle: Deterministic/Stochastic Separation

Zone	Levels	Nature	Latency	Git Protection	Hallucination Risk
Deterministic	1, 1.5, 2	Fixed rules	0-1s	Direct (no ephemeral branch)	0% (impossible)
Stochastic	3, 4, 5	LLM inference	8-15s	Protection Triad active	Mitigated to 99.99%

Fundamental insight: By resolving 60-70% of interactions in the deterministic zone, we drastically reduce both average latency and the attack surface for hallucinations.

3. Deterministic Zone: Routing Without LLM

3.1 Level 1: Bare Commands

System and git commands executed directly via JVM or ProcessBuilder. The LLM never learns the user typed anything.

Type	Pattern	Example
System	`SAFE_BARE_COMMANDS` (pwd, ls, date...)	"pwd" → Direct JVM
Git read	"git " prefix + safe subcommand	"git status" → shell
Git write	"git " prefix + safe subcommand	"git add ." → shell
Git blocked	"git " prefix + push/pull/fetch	"git push" → BLOCKED

Git command classification:

Subcommand	Risk Level	Behavior
status, log, diff, show	READ_ONLY	Direct execution
add, commit, checkout, branch, stash, init, tag	LOCAL_WRITE	Direct execution
push, pull, fetch, clone	REMOTE	BLOCKED (Ring 7)
reset --hard, clean -f	DESTRUCTIVE	BLOCKED

Performance: ~0ms. Zero tokens consumed. Zero hallucination risk.

3.2 Level 1.5: Composite Commands

Natural language intention mapping to command sequences. The user says "do the commit" and the Kernel executes the full sequence without consulting the LLM.

"do the commit"             → git add . && git commit
"commit everything"         → git add . && git commit
"do the git init and commit"→ git init && .gitignore && git add . && git commit
"save changes to git"       → git add . && git commit

Execution sequence:

executeCompositeCommit(input)
  │
  ├── Does .git exist?
  │     ├── NO + input mentions "init" → git init + auto .gitignore
  │     └── NO + no "init" → error with suggestion
  │
  ├── git add --all -- . :!.fararoni/    (excludes shadow files)
  ├── git diff --cached --stat           (verify changes)
  ├── extractCommitMessage(input)        (auto-generate or extract from quotes)
  └── git commit -m "message"

Performance: ~0ms. The full sequence (init + gitignore + add + commit) executes without LLM.

3.3 Level 2: GGUF (Simple Chat)

Casual conversation executed against an in-memory GGUF model (no network).

Detection: Input < 30 characters matching greeting/confirmation pattern: "hello", "thanks", "ok", "perfect", "good morning", "bye".

Performance: ~1 second. No tools, no git, no risk.

4. Stochastic Zone: Protection Triad

4.1 When does the stochastic zone activate?

Any input NOT captured by Levels 1, 1.5, or 2 falls to the stochastic zone. Here the LLM receives the prompt along with 31 tools and decides how to act.

"change the java version in the pom to 25"          → fs_patch
"create a REST endpoint for students"                 → fs_write
"organize the repo with feature/hotfix branches"      → GitAction (ephemeral branch)
"analyze the NullPointerException error"              → fs_read + reasoning

4.2 Protection Triad: Defense in Depth

                    ┌─────────────────────────────┐
                    │     FARARONI IRONCLAD       │
                    │  Protection Triad           │
                    └──────────────┬──────────────┘
                                   │
              ┌────────────────────┼────────────────────┐
              │                    │                    │
              ▼                    ▼                    ▼
    ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
    │   LAYER 1:       │ │   LAYER 2:       │ │   LAYER 3:       │
    │   KILL-SWITCH    │ │   GIT SAGA       │ │   SHADOW BACKUP  │
    │                  │ │                  │ │                  │
    │  Jaccard ≥ 40%   │ │  Ephemeral Branch│ │  Atomic Copy     │
    │  Volume  ≥ 50%   │ │  Auto-Revert     │ │  Pre-Write       │
    │                  │ │  Squash Merge    │ │  Recovery        │
    │                  │ │                  │ │                  │
    │  Blocks 99.9%    │ │  Contains 0.09%  │ │  Recovers 0.01%  │
    └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘
             │                    │                    │
             └────────────────────┼────────────────────┘
                                  │
                                  ▼
                    ┌─────────────────────────────┐
                    │  RESULT: 0%                 │
                    │  PERMANENT LOSS             │
                    └─────────────────────────────┘

4.3 Layer 1: Kill-Switch (Jaccard + Volume)

The Kill-Switch intercepts BEFORE each disk write and calculates two metrics:

Jaccard Index: J(A,B) = |A ∩ B| / |A ∪ B| ≥ 0.40

Volume Ratio: V = newSize / oldSize ≥ 0.50

Metric	Formula	Threshold	Detects
Jaccard Index	J(A,B) = \	A ∩ B\	/ \
Volume Ratio	V = newSize / oldSize	≥ 0.50	Massive truncations

  ORIGINAL (45 lines)             LLM PROPOSAL (20 lines)
  ──────────────────              ──────────────────────────
  class CreditoBancario {         class CreditoBancario {
    private UUID id;                private UUID id;
    private BigDecimal monto;       private BigDecimal monto;
    private BigDecimal tasaInteres; private BigDecimal tasaInteres;
    private LocalDate fecha;        // ... rest of the code
    private EstadoCredito estado; }
    private List<Pago> historial;
  }

  Volume Ratio:  20/45 = 0.44  → ✗ FAIL (< 0.50)
  Jaccard Index: 3/7  = 0.43  → ✓ PASS  (≥ 0.40)
  Decision: ✗ BLOCKED (Insufficient Volume)

4.4 Layer 2: Git Saga (Ephemeral Branches)

Selective activation: The ephemeral branch ONLY activates when all 8 conditions are met:

#	Condition	Must be met
1	Input NOT captured by Level 1, 1.5, or 2	YES
2	LLM decided to invoke GitAction (not fs_patch)	YES
3	Action is LOCAL_WRITE (add, commit, branch...)	YES
4	gitManager != null (injected in constructor)	YES
5	No ephemeral branch already active	YES
6	Is a git repo (.git exists)	YES
7	No merge/rebase in progress	YES
8	Repo has at least 1 commit (valid HEAD)	YES

LLM invokes GitAction(commit) → Level 3
  │
  ▼
ensureEphemeralBranch()
  └── git checkout -b fararoni/wip-{timestamp}
  │
  ▼
All LLM commits go to fararoni/wip-{timestamp}
User's branch remains INTACT
  │
  ▼
Finalization (squash merge):
  git checkout {original_branch}
  git merge --squash fararoni/wip-{id}
  git commit -m "[FARARONI] clean description"
  git branch -D fararoni/wip-{id}
  │
  ▼
Result: 1 single clean commit on the user's branch

4.5 Layer 3: Shadow Backups

Before each write that passes the Kill-Switch, an atomic copy is created:

.fararoni/shadow/pom.xml.v1.20260302-005551
.fararoni/shadow/pom.xml.v2.20260302-010233

These copies are the last line of defense. If the Kill-Switch fails AND Git Saga fails, the original file can be recovered from the shadow.

Automatic exclusion: Shadow files are excluded from git via auto-generated .gitignore with .fararoni/ and git add --all -- . :!.fararoni/ in the composite commit.

5. Security Link: Where does each protection act?

5.1 Activation Table by Level

Level	Kill-Switch	Ephemeral Branch	Shadow Backup	Reason
1: Bare	NO	NO	NO	No LLM, no risk
1.5: Composite	NO	NO	NO	Deterministic commands
2: GGUF	NO	NO	NO	Chat only, no writing
3: Tool Calling	YES (fs_patch)	YES (GitAction)	YES (pre-write)	Risk zone
4: Thinking	NO	NO	NO	Reasoning only
5: Fallback	NO	NO	NO	Text only

5.2 Integrated Activation Map

┌────────────────────────────────────────────────────────────────────────┐
│                  PROTECTION ACTIVATION MAP                              │
├────────────────────────────────────────────────────────────────────────┤
│                                                                        │
│  LEVEL 1 (Bare)        ○ ○ ○   No protection needed                   │
│  LEVEL 1.5 (Composite) ○ ○ ○   No protection needed                   │
│  LEVEL 2 (GGUF)        ○ ○ ○   No protection needed                   │
│                         ─────── DETERMINISTIC/STOCHASTIC FRONTIER ──── │
│  LEVEL 3 (Tool Calling) ● ● ●   Triad ACTIVE                          │
│    └─ fs_patch          ● ○ ●   Kill-Switch + Shadow                   │
│    └─ fs_write          ● ○ ●   Kill-Switch + Shadow                   │
│    └─ GitAction(status) ○ ○ ○   READ_ONLY, no protection               │
│    └─ GitAction(commit) ○ ● ○   Ephemeral branch                      │
│    └─ GitAction(push)   ✗ ✗ ✗   BLOCKED (Ring 7)                       │
│  LEVEL 4 (Thinking)    ○ ○ ○   Reasoning only                         │
│  LEVEL 5 (Fallback)    ○ ○ ○   Text only                              │
│                                                                        │
│  Legend: ● Active  ○ Inactive  ✗ Blocked                               │
└────────────────────────────────────────────────────────────────────────┘

6. Case Study: "Change the Java version to 25"

This case demonstrates how the Kernel integrates routing and protection in a real operation.

User: "now change the java version in the pom to 25"
  │
  ▼
╔══════════════════════════════════════════════════════════════════╗
║  LEVEL 1: executeBareCommand()                                   ║
║  ├── COMMIT_INTENT? → NO (doesn't contain "commit")              ║
║  ├── "git " prefix? → NO (starts with "now")                     ║
║  ├── SAFE_BARE_COMMANDS? → NO ("now" not in the set)             ║
║  └── return null                                                 ║
╚══════════════════════════════════════════════════════════════════╝
  │
  ▼
╔══════════════════════════════════════════════════════════════════╗
║  LEVEL 2: isSimpleChat? → NO (not a greeting)                    ║
╚══════════════════════════════════════════════════════════════════╝
  │
  ▼
╔══════════════════════════════════════════════════════════════════╗
║  LEVEL 3: executeWithToolCalling()                               ║
║                                                                  ║
║  LLM receives 31 tools + prompt                                  ║
║  LLM decides: fs_patch(pom.xml, "17" → "25")                     ║
║                                                                  ║
║  ┌──────────────────────────────────────────────────────┐        ║
║  │  PROTECTION TRIAD (active at Level 3)                │        ║
║  │                                                      │        ║
║  │  1. Kill-Switch:                                     │        ║
║  │     Volume: newSize/oldSize ≈ 1.0  → ✓ PASS          │        ║
║  │     Jaccard: ~0.99               → ✓ PASS            │        ║
║  │     (only changes "17" to "25", 99% identical)       │        ║
║  │                                                      │        ║
║  │  2. Shadow Backup:                                   │        ║
║  │     → .fararoni/shadow/pom.xml.v4.20260302-005551    │        ║
║  │     (pre-write copy created)                         │        ║
║  │                                                      │        ║
║  │  3. Ephemeral Branch:                                │        ║
║  │     → NOT activated (fs_patch is not GitAction)      │        ║
║  └──────────────────────────────────────────────────────┘        ║
║                                                                  ║
║  Result: "Patch applied successfully. File: pom.xml"             ║
╚══════════════════════════════════════════════════════════════════╝

After the change: "do the commit"

User: "do the commit"
  │
  ▼
╔══════════════════════════════════════════════════════════════════╗
║  LEVEL 1.5: COMMIT_INTENT matches "do.*commit"                   ║
║                                                                  ║
║  executeCompositeCommit():                                       ║
║  1. git add --all -- . :!.fararoni/   (shadow excluded)          ║
║  2. git diff --cached → pom.xml                                  ║
║  3. extractCommitMessage → "Update pom.xml"                      ║
║  4. git commit -m "Update pom.xml"                               ║
║                                                                  ║
║  ┌─────────────────────────────────────────────────────┐         ║
║  │  PROTECTION TRIAD:                                  │         ║
║  │  → NOT activated (Level 1.5 is deterministic)       │         ║
║  │  → The commit is a user operation, not the LLM's    │         ║
║  │  → Zero hallucination risk                          │         ║
║  └─────────────────────────────────────────────────────┘         ║
║                                                                  ║
║  Result: [master abc1234] Update pom.xml                         ║
╚══════════════════════════════════════════════════════════════════╝

7. Fallback for Small Models (7B)

7.1 The Problem: "JSON Leakage"

7B parameter models sometimes write tool calls as plain text instead of using the structured tool_calls field of the OpenAI response:

Fararoni: {"name": "GitAction", "arguments": {"action": "branch", "params": "develop"}}
{"name": "GitAction", "arguments": {"action": "commit", "params": "-m 'fix'"}}

The ToolExecutor never sees these because they're in content, not in tool_calls.

7.2 The Solution: extractTextToolCalls()

A parser that scans the response text looking for JSON objects with "name" + "arguments":

LLM Response (content text)
  │
  ▼
extractTextToolCalls(contentText)
  ├── Search for '{' in text
  ├── Count braces to find closure (supports nested JSON)
  ├── Parse as JSON
  ├── Verify it has "name" + "arguments"
  └── Execute via ToolExecutor

This turns a "broken" model into a functional one, without changing the model.

8. Capability Comparison

Level	Nature	Mechanism	Kill-Switch	Shadow	Latency
1: Bare	Deterministic	ProcessBuilder	NO	NO	~0ms
1.5: Composite	Heuristic	Regex + Shell	NO	NO	~0ms
2: GGUF	Local Stochastic	LLM 1.5B	NO	NO	~1s
3: Tool Calling	Stochastic/Agentic	LLM + 31 Tools	YES	YES	8-10s
4: Thinking	Reasoning	DeepSeek/Qwen3	NO	NO	10-15s
5: Fallback	Stochastic	Plain VllmClient	NO	NO	2-5s

9. Results

9.1 Latency Reduction

┌────────────────────────────────────────────────────────────────────────┐
│                    LATENCY BY OPERATION TYPE                            │
├────────────────────────────────────────────────────────────────────────┤
│                                                                        │
│  pwd (before, with LLM):   ████████████████████████████████  10s      │
│  pwd (Level 1, no LLM):    █                                 0ms      │
│                                                    Improvement: -100%  │
│                                                                        │
│  "hello" (before, w/tools): ████████████████████████████████  10s      │
│  "hello" (Level 2, GGUF):   ████                              1s      │
│                                                    Improvement: -90%   │
│                                                                        │
│  git status (before, LLM):  ████████████████████████████████  8s      │
│  git status (Level 1):      █                                 0ms      │
│                                                    Improvement: -100%  │
│                                                                        │
│  "do the commit" (before):  ████████████████████████████████  10s      │
│  "do the commit" (L 1.5):   █                                 0ms      │
│                                                    Improvement: -100%  │
│                                                                        │
└────────────────────────────────────────────────────────────────────────┘

9.2 Integrity Protection

┌────────────────────────────────────────────────────────────────────────┐
│                    PROTECTION EFFECTIVENESS                            │
├────────────────────────────────────────────────────────────────────────┤
│                                                                        │
│  Layer 1 (Kill-Switch):                                               │
│  ████████████████████████████████████████████████████████  99.9%      │
│  Blocked by Jaccard/Volume                                            │
│                                                                        │
│  Layer 2 (Git Saga):                                                  │
│  ████████████████████████████████████████████████████████  99.99%     │
│  Contained via ephemeral branch + auto-revert                         │
│                                                                        │
│  Layer 3 (Shadow Backup):                                             │
│  ██████████████████████████████████████████████████████████  100%     │
│  Recovered from shadow files                                          │
│                                                                        │
│  RESULT: 0 PERMANENT LOSSES                                           │
└────────────────────────────────────────────────────────────────────────┘

10. Conclusion

The Fararoni Kernel demonstrates that secure and efficient LLM integration in software development requires a hybrid architecture that combines:

Intelligent Routing: 60-70% of interactions are resolved in the deterministic zone (Levels 1-2), eliminating latency and hallucinations for operational tasks.
Defense in Depth: The remaining 30-40% passes through the Protection Triad, where each layer captures escapes from the previous one until reaching 0% permanent loss.
Small Model Adaptability: The text fallback (extractTextToolCalls) enables using 7B parameter models that don't properly handle the tool calling protocol, democratizing access to these capabilities.

The explicit separation between the deterministic zone and the stochastic zone is not just a performance optimization: it is a security principle. By making the LLM "never know" about trivial commands, we eliminate the widest attack surface. And by shielding the points where the LLM DOES operate, we guarantee that its stochastic nature cannot cause permanent damage.

References

Cruz, E. (2026). Fararoni Ironclad: Deterministic Guardrails for Code. Technical Report v3.
OWASP LLM Top 10 (2025). Security Risks in Large Language Model Applications.
IEEE/ACM ICSE (2025). Proceedings on AI-Assisted Software Engineering.

## Try It

  brew tap ebercruzf/fararoni && brew install fararoni

Also available as standalone binaries for macOS, Linux & Windows.

About the Author

Eber Cruz Fararoni is a software engineer with a decade of experience designing backend infrastructure and distributed systems.

Currently focused on AI-assisted software engineering, deterministic guardrails, and hybrid kernel architectures for secure LLM execution.

This article documents the architecture behind C-FARARONI, an experimental ecosystem for technological

sovereignty and secure local AI model execution.

LinkedIn · GitHub · ebercruz.com

Canonical source: fararoni.dev/publicacion/kernel-hibrido

Sovereign Intelligence on Apple Silicon: Breaking the Microsecond Barrier with Java 25 and Panama FFM

Eber Cruz Fararoni — Tue, 24 Mar 2026 13:58:34 +0000

By Eber Cruz | March 2026

The audio engine runs two completely independent TTS backends, both executing inference on the Metal GPU but with fundamentally different architectural paths.

If you've ever tried to build a truly conversational AI, you know that latency is the enemy of presence. It's not just about how fast the model generates tokens; it's about how fast the system can "yield the floor" when a human starts to speak.

Standard Java audio stacks and JNI bridges often introduce non-deterministic delays that make real-time, full-duplex interaction feel robotic. To solve this for the C-Fararoni ecosystem, I decided to bypass the legacy abstractions and talk directly to the silicon.

In this deep dive, I share the architecture and real-world benchmarks of a system built on Java 25, Panama FFM, and Apple Metal GPU. We aren't talking about millisecond improvements here—we've measured a playback interrupt cycle that completes in just 833 nanoseconds.

What's inside:

Zero-JNI Architecture: How we achieved a 42ns overhead using the Foreign Function & Memory API.
Metal GPU Orchestration: Running 0.6B and 1.7B neural models locally on 32 GPU cores via PyTorch MPS and ggml-metal.
The "Abort" Benchmark: Why a 6,000x improvement over our initial latency target was necessary for Sovereign AI.

Bye Bye JNI: Metal GPU, CoreAudio and Panama FFM on Apple Silicon

When we set out to build a voice-first AI assistant that could hold a real conversation — not the kind where you wait three seconds for a response, but the kind where the system knows when to stop talking the instant you speak — we realized the entire Java audio stack had to go. No JNI. No abstraction layers. Just Java talking directly to the hardware through Panama FFM, CoreAudio rendering audio at 24kHz mono float32 through a native callback, and Metal GPU running neural inference on all 32 cores of an M1 Max.

This is the architecture behind Fararoni's audio engine. Every number in this document was measured on real hardware, on real code running in a high-fidelity development environment. These are not theoretical projections — they are measurements taken directly from Fararoni's core as we build the foundation for a sovereign, low-latency AI.

What follows is the story of three bridges: Java to native code in 42 nanoseconds, a playback interrupt in 833 nanoseconds, and two neural models — 0.6B and 1.7B parameters — running Metal compute kernels to synthesize human voice.

Direct Metal: How We Talk to the Hardware

The foundation of the audio engine is a C++ library (fararoni_audio.cpp) that programs CoreAudio's AudioUnit directly. No wrappers, no middleware. The output unit is configured at 24kHz mono float32 with a render callback that copies PCM samples via zero-copy memcpy — the audio thread owns the buffer, and we never fight it for a lock.

The critical path, however, is not playback — it's interruption. In a conversational AI, the system must stop speaking the instant the user starts talking. That means the abort command has to be fast. Not "fast for software" — fast enough that the hardware is the bottleneck.

Here is the entire abort implementation:

extern "C" void fararoni_abort_playback(void) {
    AudioUnitReset(outputUnit, kAudioUnitScope_Global, 0);  // <1us measured
    AudioOutputUnitStop(outputUnit);
    playbackCtx.position = playbackCtx.size;
    playbackCtx.finished = true;
    g_playing.store(false);
}

AudioUnitReset flushes the AudioUnit's internal buffers. On Apple Silicon this is sub-microsecond because there's no DMA transfer to wait for — the buffer lives in unified memory — no lock contention (AudioUnit runs on its own thread), and the buffer is small (24kHz × buffer_size_frames × 4 bytes).

We measured it. On an M1 Max running Java 25.0.1:

Zero-Overhead Bridge: The jump from Java to C++ via Panama FFM adds just 42ns (P50).
Hardware Reset: The AudioUnitReset command halts the CoreAudio engine in 459ns (P50).
End-to-End: The complete abort cycle consolidates at 833ns (P50) — 0.0008ms — beating the original 5ms target by 6,000x. Even at P99 (8.4µs), it's 600x under target.

An important distinction: this measures the interrupt command — the ability to stop audio that is already playing. It is not the latency of generating audio, which takes seconds. The only physical bottleneck that remains is the microphone buffer itself: the AudioUnit HAL input needs ~5-10ms to fill a capture buffer, and no software can change that. Once the buffer arrives, our software reacts in under 2 microseconds.

Frameworks linked directly (Makefile):

CoreFoundation, CoreAudio, AudioToolbox, AudioUnit, IOKit,
Metal, MetalKit, Accelerate, Foundation

Dual-Engine TTS: Direct vs. Indirect Metal GPU Execution

The audio engine runs two completely independent TTS backends, both executing inference on the Metal GPU but with fundamentally different architectural paths.

The 0.6B Engine: ggml-metal Compute

The fast path uses qwen3-tts-cli, a C++ binary that loads a GGUF-quantized 0.6B model and runs inference through ggml-metal — a complete Metal compute implementation that compiles .metal shaders at runtime, creates MTLComputePipelineState for each tensor operation, and dispatches MTLComputeCommandEncoder with optimized thread groups. The buffers live in unified memory: zero-copy between CPU and GPU.

To achieve this efficiency, the engine relies on highly optimized Metal implementations of core neural network operations. These "compute shaders" include:

Attention: The mechanism that allows the model to dynamically weight the importance of different parts of the input text sequence when generating each audio frame.

Softmax: The activation function that normalizes raw model scores into a probability distribution, crucial for accurate audio token selection.

RoPE (Rotary Positional Embeddings): An advanced method for encoding token positions in the sequence, improving the model's understanding of context and order compared to traditional absolute embeddings.

Java launches it as a subprocess:

List<String> cmd = List.of(
    binaryPath.toString(),       // qwen3-tts-cli
    "-m", modelDir.toString(),   // GGUF model dir
    "-t", styledText,            // text to synthesize
    "-o", outputWav.toString()   // WAV output
);
Process proc = new ProcessBuilder(cmd).start();

For a single word ("Hi."), the 0.6B engine produces 0.3 seconds of audio in 3.7s total (including model load). For longer text, autoregressive generation scales linearly with audio duration — 10 words producing 2.8s of audio take ~32s.

The 1.7B Engine: PyTorch MPS on Metal

The high-fidelity path runs a persistent Python sidecar (tts_server.py) with three variants of Qwen3-TTS-12Hz-1.7B loaded into GPU memory via PyTorch's MPS backend. MPS (Metal Performance Shaders) translates PyTorch tensor operations into Metal compute commands — the same MTLComputeCommandEncoder, the same unified memory buffers, the same GPU cores.

def detect_device():
    if torch.backends.mps.is_available():
        return "mps"            # Apple Silicon -> Metal GPU
    elif torch.cuda.is_available():
        return "cuda"
    else:
        return "cpu"

m = Qwen3TTSModel.from_pretrained(model_id, device_map="cpu")
m = m.to("mps")                # Tensors move to Metal GPU

The sidecar stays resident — the model loads once and serves requests via a JSON-line protocol over stdin/stdout. Java orchestrates:

Java (PythonSidecarBackend)
  -> stdin: {"command":"synthesize", "speaker":"Aiden", "text":"..."}
  -> Python: PyTorch MPS -> Metal GPU (1.7B inference)
  -> stdout: {"status":"ok", "wav":"/tmp/fararoni_tts_xxx.wav"}

Python is only the invocation wrapper. The heavy lifting — neural inference — runs 100% on the Metal GPU. We use Python because HuggingFace Transformers publishes Qwen3-TTS models with a Python API, and PyTorch MPS is the bridge to Metal. The data never leaves the machine.

Why two engines? The 0.6B model is instant-quality: fast, stateless, no persistent process. The 1.7B model is studio-quality: speaker embeddings, richer prosody, but requires a warm sidecar. The routing engine (FararoniAudioEngine.synthesizeToFile()) selects the backend based on speaker availability and quality preference — builtin speakers (Aiden, Dylan, Vivian, Eric) route to the 1.7B sidecar, while unknown speakers fall back to the 0.6B CLI.

Aspect	0.6B (ggml-metal)	1.7B (PyTorch MPS)
Metal path	.metal shaders compiled at runtime	MPS precompiled kernels
Model format	GGUF quantized	HuggingFace float16/32
Java interface	ProcessBuilder (per-invocation)	stdin/stdout JSON-line (persistent)
Speaker selection	No (CLI limitation)	Yes (speaker embedding)
Quality	Instant	Studio
Both execute on	Metal GPU compute	Metal GPU compute
Hardware Access	Direct Metal (zero framework overhead)	Indirect Metal (PyTorch/Python tax)

High-Fidelity Synthesis: Scaling to 1.7B with PyTorch MPS

The 1.7B sidecar is where data sovereignty meets quality. The model runs locally on 32 GPU cores — no cloud API, no network hop, no third-party data processing. For an AI assistant that handles private conversations, this is not a feature; it's a requirement.

Measured synthesis times on M1 Max (all routed to 1.7B Python sidecar via Metal GPU):

Studio quality — speaker-embedded, full prosody:

Speaker	Text	Audio Duration	Total Time
Aiden	"Hello, I am Aiden..." (10 words)	3.9s	47.5s
Dylan	"Hey, I am Dylan..." (10 words)	3.4s	40.1s
Vivian	"Hola, soy Vivian..." (8 words)	3.3s	38.9s
Eric/Marcus	"Buenos dias, soy Marcus..." (12 words)	4.4s	52.0s

Instant quality — still 1.7B for builtin speakers:

Speaker	Text	Audio Duration	Total Time
Aiden	"Hello, I am Aiden..."	2.6s	30.3s
Vivian	"Hola, soy Vivian..."	2.6s	30.9s

0.6B Metal — unknown speakers, activeBackend fallback:

Speaker	Text	Audio Duration	Total Time
(unknown, 10 words)	"Hello world..."	2.8s	32.3s
(unknown, 9 words)	"Buenos dias..."	2.6s	31.7s
(unknown, 1 word)	"Hi."	0.3s	3.7s

These are real numbers from real synthesis runs, not estimates. The routing was verified by tracing the six conditions in FararoniAudioEngine.synthesizeToFile() (line 755-802).

Zero JNI: Panama FFM as the Universal Bridge

Every call from Java to native code in this engine goes through Panama FFM (JEP 454). Zero JNI imports. Zero generated headers. Zero boilerplate.

The pattern is the same across all three native-bridging classes:

Linker linker = Linker.nativeLinker();
SymbolLookup nativeLib = SymbolLookup.libraryLookup(path, Arena.global());
MethodHandle fn = linker.downcallHandle(
    nativeLib.find("fararoni_xxx").get(),
    FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.ADDRESS)
);
int result = (int) fn.invokeExact(memorySegment);

Three classes, three domains, one pattern:

NativeAudioPlayer — Playback control. Four downcall handles: initEngine, playBuffer, stopEngine, isInitialized. The buffer transfer is zero-copy via confined arenas:

try (Arena arena = Arena.ofConfined()) {
    MemorySegment nativeBuffer = arena.allocate(ValueLayout.JAVA_FLOAT, samples.length);
    MemorySegment.copy(samples, 0, nativeBuffer, ValueLayout.JAVA_FLOAT, 0, samples.length);
    int result = (int) playBuffer.invokeExact(nativeBuffer, samples.length);
}

Note on Manual Memory (Arenas): In high-performance Java, an Arena is a bounded memory region that allows for deterministic, off-heap allocation. Unlike standard Java objects managed by the Garbage Collector, memory within an Arena is orchestrated manually. This ensures that our 833ns critical path remains GC-free, providing the microsecond-level determinism required for real-time conversational AI.

Arena.ofConfined() gives us deterministic memory: allocated before the call, freed when the try-with-resources block ends. No GC pressure, no finalizers, no surprises. Measured allocation+copy for 1 second of audio (24,000 float32 samples): 5.3µs (P50). For 100 seconds of audio: 434µs.

WhisperEngine — Engine control and STT. Eight downcall handles spanning both the TTS abort path (abortPlayback, setVolume, isPlaying, initEngine, getTelemetry) and Whisper STT for voice commands (whisperInit, startTranscription, stopTranscription). This class also demonstrates Panama upcalls — C-to-Java callbacks:

MemorySegment callbackStub = linker.upcallStub(
    MethodHandles.lookup().bind(handler, "onTranscript", ...),
    FunctionDescriptor.ofVoid(ValueLayout.ADDRESS, ValueLayout.ADDRESS),
    callbackArena
);

VadDetector — Voice Activity Detection. Four handles: vadIsSpeech, rmsEnergy, startVadCapture, stopEngine. The VAD runs inline on the audio capture thread — no thread hop, no queue.

The measured FFM overhead: 42ns per downcall (P50), based on 10,000 iterations of a noop function. The JEP 454 spec claims ~10ns; the 4x difference is explained by nanoTime granularity on M1 (42ns resolution), branch prediction variability, and cache state. Still sub-microsecond, still negligible for audio work.

The Anatomy of a Sub-Microsecond Interrupt

Full-duplex means the system listens while it speaks. When the user starts talking mid-sentence, the interrupt chain fires:

User speaks while TTS plays audio
  +-- HAL AudioUnit captures mic -> callback
      +-- RMS energy > threshold -> speech detected
          +-- Panama upcall: C -> Java (~42ns)
              +-- WhisperEngine.abortPlayback()
                  +-- Panama downcall: Java -> C (~42ns)
                      +-- fararoni_abort_playback()
                          +-- AudioUnitReset: audio stops (459ns)
                          +-- AudioOutputUnitStop

Three measured segments tell the whole story:

The Bridge (Panama FFM round-trip): upcall + downcall = ~84ns. Java is not a bottleneck. The foreign function boundary is invisible at audio timescales.

The Command (AudioUnitReset + stop on C side): 459ns (P50). The AudioUnit flushes its buffers in unified memory — no DMA, no contention.

The Full Cycle (Java → Panama → C → AudioUnitReset → C → Panama → Java): 833ns (P50). Under one microsecond. The original design target was 5ms; we beat it by 6,000x.

The one thing software cannot accelerate is physics. The microphone's AudioUnit HAL input buffer takes ~5-10ms to fill — a hardware constraint determined by buffer_size_frames, not by code. Once that buffer delivers the speech event, our stack reacts in under 2 microseconds. The total real-world interrupt latency is dominated entirely by the microphone hardware, not by the software chain.

Real-World Synthesis: Measured, Not Estimated

Every claim in this document traces back to NativeAudioBenchmark.java, a standalone benchmark class that exercises the native bridge through Panama FFM on a live libfararoni_audio.dylib.

The benchmark measures six distinct operations across thousands of iterations on an M1 Max with Java 25.0.1:

FFM downcall overhead: 10,000 calls to a noop — 42ns (P50), 292ns (P99)
Abort C-side (AudioUnitReset + Stop): 100 calls — 459ns (P50), 2,750ns (P99)
Abort end-to-end (Java→C→Java): 100 calls — 833ns (P50), 8,375ns (P99)
Arena alloc+copy (1s audio, 24K samples): 5.3µs (P50)
Arena alloc+copy (100s audio, 2.4M samples): 434µs (P50)
Native buffer read (1s audio): 22.4µs (P50)

The synthesis times are equally real — every speaker, every quality level, every backend was tested through the REST endpoint (POST /v1/audio/synthesize) with the routing verified by tracing the condition branches in FararoniAudioEngine.

What we do not claim: we do not claim "5ms audio generation latency." Generation is neural inference and takes seconds. What is sub-microsecond is the command to stop — and that distinction matters, because it's the difference between an assistant that talks over you and one that yields the floor instantly.

Why This Matters

An AI assistant that can generate speech is not the same as one that can hold a conversation. Conversation requires knowing when to stop. Not "stop after a timeout" — stop now, mid-phoneme, because the human on the other end just opened their mouth.

That's what 833 nanoseconds buys us. Not speed for speed's sake, but the foundation for an AI that doesn't just respond — it knows when to be silent and listen. Full-duplex interrupt is the mechanical prerequisite for conversational presence: the system must be able to yield the floor faster than a human can perceive the delay.

The architecture we've built — Panama FFM as the zero-overhead bridge, CoreAudio's AudioUnit as the render engine, Metal GPU driving two neural models, and a sub-microsecond abort chain — is not about showing off low-level programming. It's about removing every artificial barrier between the AI and natural conversation, so the only latency that remains is the physics of a microphone filling its buffer.

Everything runs on-device. The voice models, the inference, the audio rendering, the interrupt — all local, all on Metal, all without a single byte leaving the machine. For an assistant that handles private conversations, sovereignty over the audio pipeline is not optional.

**The Zero-GC Determinism Factor

While industry-standard frameworks like PyTorch provide incredible flexibility, they often carry a 'latency tax' due to their Python-heavy orchestration and complex abstraction layers. By leveraging Java 25’s Scoped Arenas, we’ve moved the critical path Off-Heap. This means the Garbage Collector never touches our 833ns interrupt logic. We aren't just calling a model; we are orchestrating silicon without the overhead of the giants.

Appendix: Raw Benchmark Data

Environment: Java 25.0.1 | aarch64 | Mac OS X | Apple M1 Max

========================================================================
  FARARONI AUDIO BENCHMARK — Panama FFM + CoreAudio + Metal
========================================================================

[System.nanoTime() overhead]
  Iterations: 10,000
  Mean:           9 ns  |  P50:   0 ns  |  P99:  42 ns  |  Max: 167 ns

[FFM Downcall Overhead (noop)]
  Iterations: 10,000
  Mean:          88 ns  |  P50:  42 ns  |  P99: 292 ns  |  Max: 22,916 ns

[Arena alloc+copy (1024 samples = 0.04s audio)]
  Mean:       4,916 ns  |  P50: 3,917 ns  |  P99: 28,125 ns

[Arena alloc+copy (24000 samples = 1.00s audio)]
  Mean:       5,515 ns  |  P50: 5,334 ns  |  P99:  9,583 ns

[Arena alloc+copy (240000 samples = 10.00s audio)]
  Mean:      35,841 ns  |  P50: 34,791 ns |  P99: 54,584 ns

[Arena alloc+copy (2400000 samples = 100.00s audio)]
  Mean:     434,339 ns  |  P50: 432,083 ns | P99: 471,959 ns

[Native buffer read (24000 floats = 1s audio)]
  Mean:      22,970 ns  |  P50: 22,375 ns |  P99: 30,875 ns

[Abort Playback (C-side only: AudioUnitReset+Stop)]
  Iterations: 100
  Mean:         514 ns  |  P50: 459 ns  |  P99: 2,750 ns  |  Max: 2,750 ns

[Abort Playback (Java->C->AudioUnitReset->Java)]
  Iterations: 100
  Mean:       1,016 ns  |  P50: 833 ns  |  P99: 8,375 ns  |  Max: 8,375 ns
========================================================================

Benchmark executed with NativeAudioBenchmark.java on libfararoni_audio.dylib
compiled with fararoni_benchmark.cpp (Makefile updated, make && make install).
M1 Max, 32-core GPU, Java 25.0.1, 2026-03-21.

## Try It

  brew tap ebercruzf/fararoni && brew install fararoni

Also available as standalone binaries for macOS, Linux & Windows.

Sovereign AI Infrastructure: Scaling Enterprise Agents from 8GB RAM to Global Clusters with Fararoni.

Eber Cruz Fararoni — Sat, 21 Mar 2026 20:48:18 +0000

The Era of Local Execution

AI deployment has shifted from cloud experimentation to the urgent need for Edge Sovereignty. As global giants like Alibaba (Qwen) and Huawei (Ascend) release increasingly powerful open-weight models, enterprises face a critical bottleneck: How do we execute these agents securely, privately, and on existing hardware?

Fararoni was born to bridge this gap, turning agent orchestration from a data center luxury into a native capability of any standard office computer.

1. Hardware Democratization: Enterprise AI on 8GB of RAM

Most AI infrastructures require expensive GPUs and nightmare software configurations. Fararoni breaks this barrier:

Extreme Efficiency: Capable of running a full WhatsApp or Telegram service flow using only 8GB or 16GB of RAM.
Optimized for Qwen: Specifically designed to leverage models like Qwen 1.5B/7B, allowing companies to onboard users into AI Agents without investing in new hardware.
"Zero-Config" Installation: A single binary. No Python, no Docker, no dependency hell. Ideal for mass deployment in restricted corporate environments.
Immediate Use Case: A basic office server can now manage a 24/7 Customer Service WhatsApp Agent, processing data locally with total privacy.

2. The "Rabbit-Turtle" Architecture

To maximize efficiency on modest hardware, Fararoni implements a hybrid computing strategy:

The Rabbit (The Orchestrator): A lightweight local model (e.g., Qwen 1.5B) that handles fast interactions, message filtering, and routine tasks in milliseconds.
The Turtle (The Thinker): An orchestrator that scales to heavier models (7B, 32B, or external APIs like Claude/DeepSeek) only when the task's complexity demands it.

This ensures a fluid user experience even on limited hardware, drastically optimizing cost-per-token and energy consumption.

The Fararoni Deployment Matrix: Scaling from 8GB Edge devices with Qwen 1.5B to High-Density Sovereign Clusters with MoE models.

3. The Nervous System: NATS and Data Sovereignty

For organizations requiring strict security compliance, Fararoni offers:

Event Bus (NATS): Total decoupling that allows agents to live on different nodes, ensuring sensitive data never leaves the secure perimeter.
DAG-Based Traceability: Every decision made by the AI is recorded in a Directed Acyclic Graph. It is auditable, transparent, and predictable.
Apache 2.0 License: The gold standard for industrial collaboration in both the East and West, allowing integration into commercial products without legal risks.

4. Strategic Alignment: Why Fararoni is the Partner for Giants

For the Alibaba/Qwen Ecosystem: Fararoni is the ideal "transport layer" to bring Qwen models to the end-user's desktop and SMEs, facilitating massive model adoption.
For Huawei Hardware (Ascend/Kunpeng): As an architecture based on native binaries and memory efficiency, Fararoni aligns perfectly with "Technological Decoupling" and total stack control strategies.
For European Sovereignty (GAIA-X): We offer total control over data flow, eliminating dependence on third-party "black boxes."

Conclusion: Start Small, Scale Infinitely.

The true revolution isn't the biggest model; it’s the agent that is exactly where the user needs it. With Fararoni, you can start today by installing an agent on an 8GB laptop and end tomorrow orchestrating a sovereign swarm on a national cluster.

The era of agents is here, but the real revolution is executing them with sovereignty.

About the Author

This article documents the architecture behind C-FARARONI, an experimental ecosystem for technological

sovereignty and secure local AI model execution.

LinkedIn · GitHub · ebercruz.com

🔗 Immediate Action

## Try It

  brew tap ebercruzf/fararoni && brew install fararoni

Also available as standalone binaries for macOS, Linux & Windows.

Download Installer: fararoni.dev (Windows, Mac, Linux).
Test the WhatsApp Sidecar: Integrate AI into your communication flow in under 5 minutes.
License: Apache 2.0 – Your infrastructure, your rules.

Moving Beyond Chatbots: Architecting a Sovereign AI Ecosystem with Java 25 & NATS

Eber Cruz Fararoni — Wed, 18 Mar 2026 13:04:22 +0000

🇪🇸 Lee la versión en español aquí

The Problem: The Erosion of Technological Sovereignty

As engineers, we've fallen into a trap of convenience. We are building on "quicksand": closed APIs, black boxes, and total cloud dependency. Every time we send a prompt, we give away context and lose sovereignty.

I decided I didn't want an "assistant" to tell me jokes. I wanted a command infrastructure. Thus, Fararoni was born, an ecosystem designed to treat AI as what it should be: executable infrastructure, not just a chat interface.

1. The Centurion Vision: Command Architecture

In Fararoni, we move away from the "copilot" model to adopt the Centurion Vision.

The Human is the Architect: Defines the strategy, limits, and mission objective.
The AI is the Centurion: Orchestrates and executes.

To achieve this, the architecture cannot be linear. We need an infrastructure that supports failures, latency, and real-time context switching.

2. The Tech Stack: Why Java 25 and NATS?

Java 25 and the Power of Virtual Threads

Many ask: Why not Python? The answer is simple: concurrency and robustness.
By using Java 25, we leverage virtual threads (Project Loom) to handle hundreds of agents and processes in the "Swarm" lightly. With GraalVM support, we achieve native binaries that boot in milliseconds, ideal for a CLI that must feel instantaneous.

NATS: The Nervous System

We don't use an internal REST API for module communication. We use NATS as an event bus. This allows us to:

Total decoupling: The "Sidecars" (WhatsApp, Telegram, Terminal) don't know who processes the order; they only listen to the bus.
Resilience: If a local model goes down, the message stays on the bus until a worker is ready.

3. Tactical Innovation: DAGs and Hot-Swapping

Traceability via DAGs (Directed Acyclic Graphs)

AI is often a black box. In Fararoni, every AI decision is mapped in a DAG. This allows the human architect to audit the flow:

Where did this information come from?
Which model made the decision?
What was the cost and latency?

If it's not auditable, it's not professional.

Model Hot-Swap: The Bridge Between Local and Cloud

One of the biggest challenges was the Hot-Swap.

For low-sensitivity tasks or pre-processing, we use a local 1.5B parameter model.
If the task scales in complexity, the system does a "Hot-Swap" to Claude 3.5 or GPT-4 without losing the mission state.

You maintain control over what data leaves your infrastructure and what doesn't.

4. Digital Heritage and Open Source

I have released the communication core and plugins under the Apache 2.0 license. I'm not looking to create another captive platform; I want to help other engineers in Latin America and the world build their own Digital Heritage.

Sovereignty isn't just a pretty word; it's having the binaries, the data bus, and the models under your own command.

Conclusion: The Swarm is Growing

Fararoni is already real. The installers are available, and the terminal is already orchestrating missions.
It is not a finished product, it's a living ecosystem.

What do you think about using NATS for LLM orchestration compared to traditional queue-based architectures like RabbitMQ or Kafka? I'll read you in the comments.

Explore the code and documentation at: fararoni.dev

## Try It

  brew tap ebercruzf/fararoni && brew install fararoni

Also available as standalone binaries for macOS, Linux & Windows.

*Tags: #OpenSource #Java25 #GraalVM #Java #ProjectLoom #CloudNativeJava #SelfHosted #NATS #SoftwareArchitecture #SovereignAI #SoftwareArchitecture #Fararoni #AI #TechSovereignty #Ollama #DeepSeek #Qwen #LocalLLM #AIInfrastructure

Más allá de los Chatbots: Construyendo un Ecosistema de IA Soberana con Java 25 y NATS

Eber Cruz Fararoni — Wed, 18 Mar 2026 12:59:59 +0000

🇬🇧 Read the English version here

El Problema: La erosión de la Soberanía Tecnológica

Como ingenieros, hemos caído en una trampa de conveniencia. Estamos construyendo sobre "arenas movedizas": APIs cerradas, cajas negras y una dependencia total de la nube. Cada vez que enviamos un prompt, regalamos contexto y perdemos soberanía.

Decidí que no quería un "asistente" que me contara chistes. Quería una infraestructura de mando. Así nació Fararoni, un ecosistema diseñado para tratar a la IA como lo que debería ser: infraestructura ejecutable, no solo una interfaz de chat.

1. La Visión del Centurión: Arquitectura de Mando

En Fararoni, nos alejamos del modelo de "copiloto" para adoptar la Visión del Centurión.

El Humano es el Arquitecto: Define la estrategia, los límites y el objetivo de la misión.
La IA es el Centurión: Orquesta y ejecuta.

Para lograr esto, la arquitectura no puede ser lineal. Necesitamos una infraestructura que soporte fallos, latencia y cambios de contexto en tiempo real.

2. El Stack Técnico: ¿Por qué Java 25 y NATS?

Java 25 y el poder de los Virtual Threads

Muchos se preguntan: ¿Por qué no Python? La respuesta es simple: concurrencia y robustez.
Al usar Java 25, aprovechamos los hilos virtuales (Project Loom) para manejar cientos de agentes y procesos del "Enjambre" (Swarm) de manera ligera. Con el soporte de GraalVM, logramos binarios nativos que arrancan en milisegundos, ideales para una CLI que debe sentirse instantánea.

NATS: El Sistema Nervioso

No usamos una API REST interna para la comunicación de módulos. Usamos NATS como bus de eventos. Esto nos permite:

Desacoplamiento total: Los "Sidecars" (WhatsApp, Telegram, Terminal) no saben quién procesa la orden, solo escuchan el bus.
Resiliencia: Si un modelo local se cae, el mensaje permanece en el bus hasta que un trabajador (worker) esté listo.

3. Innovación Táctica: DAGs y Hot-Swapping

Trazabilidad mediante DAGs (Directed Acyclic Graphs)

La IA suele ser una caja negra. En Fararoni, cada decisión de la IA se mapea en un DAG. Esto permite que el arquitecto humano audite el flujo:

¿De dónde vino esta información?
¿Qué modelo tomó la decisión?
¿Cuál fue el costo y la latencia? Si no es auditable, no es profesional.

Hot-Swap de Modelos: El puente entre lo Local y la Nube

Uno de los mayores retos fue el Cambio en Caliente.

Para tareas de baja sensibilidad o pre-procesamiento, usamos un modelo local de 1.5B parámetros.
Si la tarea escala en complejidad, el sistema hace un "Hot-Swap" a Claude 3.5 o GPT-4 sin perder el estado de la misión. Tú mantienes el control de qué datos salen de tu infraestructura y cuáles no.

4. Patrimonio Digital y Open Source

He liberado el núcleo de comunicación y los plugins bajo la licencia Apache 2.0. No busco crear otra plataforma cautiva; busco ayudar a otros ingenieros en Latinoamérica y el mundo a construir su propio Patrimonio Digital.

La soberanía no es solo una palabra bonita; es tener los binarios, el bus de datos y los modelos bajo tu propio mando.

Conclusión: El Enjambre está creciendo

Fararoni ya es real. Los instaladores están disponibles y la terminal ya orquesta misiones.
No es un producto terminado, es un ecosistema vivo.

¿Qué opinan sobre el uso de NATS para orquestación de LLMs en comparación con arquitecturas tradicionales basadas en colas como RabbitMQ o Kafka? Los leo en los comentarios.

Explora el código y la documentación en: fararoni.dev

From Startup to Unicorn: A Blueprint for Secure Enterprise Architecture

Eber Cruz Fararoni — Tue, 13 Jan 2026 02:22:16 +0000

How to implement a Reactive Security Flow with Spring Boot, Redis, and JWT for high-scale environments avoiding the Microservices Trap.

1. The Context: Speed vs. Stability

Startups often face a dilemma: build an MVP fast to validate the market, or build for scale to handle future growth. The “move fast and break things” approach works for a month, but creates technical debt that kills growth in Year 2.
In the Fintech space, you don’t have the luxury of “breaking things.” You need the speed of a startup but the resilience and security of a bank.

2. The Architecture: The Hybrid Approach

Instead of jumping straight into a complex Microservices mesh (which drains budget and requires a DevOps army) or staying in a Monolith (which doesn’t scale), I propose a Modular Hybrid Architecture.
This approach decouples the Security Layer (Reactive) from the Business Logic (Transactional), allowing us to deploy on Serverless platforms (like Cloud Run) while keeping costs low.

System Overview (The Ecosystem)

3. Key Decision: The Multi-Schema Database Strategy

One of the biggest mistakes startups make is spinning up a new RDS instance for every microservice. This burns money.
The Solution: A single PostgreSQL instance with Logical Isolation via Schemas.
Why: It strictly enforces domain boundaries (Business, Payment, Security) without the overhead of managing 10 different database servers.
The Benefit: We can perform cross-schema joins for analytics when needed, but the application code treats them as separate data sources. This prepares us for a physical split in the future ("One-to-N" scaling) without refactoring logic.

4. The Security Core: Reactive Gateway + Servlet Logic

As shown in the diagram above, we implemented a strict separation of concerns:

The Reactive Shield (Spring Cloud Gateway): Handles high concurrency, manages the SSL termination, and validates the JWT signature before the request ever touches the business logic.
The Business Core (Servlet): Once the request is safe, it passes to the blocking transactional services where complex business logic lives.
State Management (Redis): We use a “Redis Blacklist” pattern to allow instant token revocation — fixing the main security flaw of stateless JWTs.

5. Why “HttpOnly” Cookies?

We moved away from storing tokens in LocalStorage (vulnerable to XSS) to HttpOnly Secure Cookies. This ensures that even if a malicious script runs on the client, it cannot exfiltrate the user’s credentials. This is a non-negotiable standard for Fintech applications.

6. Future-Proofing: The Path to Apigee**

A critical aspect of this architecture is Cost-Efficiency. Startups cannot afford expensive Enterprise API Management licenses from Day

Current State (Lean): We use Spring Cloud Gateway to handle standard concerns like Basic Rate Limiting, CORS, and Auth Validation. This runs on Cloud Run with minimal cost.
Future State (Enterprise): As the business succeeds and traffic spikes (“One-to-N”), we don’t need to refactor. We can simply place Google Apigee in front of our Gateway.

This allows us to offload advanced security features — such as DDoS protection, KVM (Key Value Maps), IP Whitelisting, and complex Quotas — to a dedicated layer, keeping our core services lightweight and focused purely on business logic.

Conclusion

This architecture is not just code; it is a business asset. It allows a small team of 3 engineers to handle traffic that usually requires a team of 20, keeping the burn rate low while maintaining banking-grade security.
I am currently exploring new opportunities to apply these architectural patterns at an Enterprise scale. If you are looking for a Staff Engineer focused on Security and Scalability, let’s connect on LinkedIn or check my Portfolio.

Weaviate for RAG: When It Shines (and When It Doesn’t)

Eber Cruz Fararoni — Mon, 15 Dec 2025 00:28:06 +0000

A hands-on review after building an enterprise-grade PoC — not just another “Hello World”

As a Technical Lead & AI Architect (Hands-On) with a focus on RAG Systems and experience building solutions for organizations like HSBC, Scotiabank, and CFE, I'm always evaluating cutting-edge technologies. Recently, at AI Research Lab in Mexico City (Feb 2025 – Jun 2025), I spearheaded the architecture for a comprehensive Retrieval Augmented Generation (RAG) solution for an internal Business Intelligence Engine PoC. This was not a client-facing product, but a technical deep-dive to validate architecture, latency, and security patterns for future enterprise deployment. The PoC was designed to rigorously test RAG architectures for real-world readiness, incorporating:

Full enterprise patterns (auth, error handling, observability)

Local LLMs (DeepSeek-R1 via Ollama)

100% data sovereignty

Benchmarks on real hardware (GCP n2-standard-8)

My contributions included designing a multi-layered RAG architecture with reactive streaming patterns (Spring WebFlux, Project Reactor), architecting Weaviate v4 integration with optimized Sentence-BERT embeddings for financial document processing, and directing the local LLM integration strategy — leveraging my background as a Google Certified GenAI Leader.

🔗 Full architecture details: ebercruz.com/technical
💻 Code (MIT, non-commercial): github.com/ebercruzf/enterprise-intelligence-engine

✅ Where Weaviate Delivers Value — in practice

1. Hybrid Search: `nearText` + `where` = Fewer False Positives

In real use, users don’t ask clean questions like “summarize Q3 earnings”. They often phrase queries like:

“What did the compliance team say about loan approvals last quarter?”

Most vector DBs force a choice between semantic or keyword search. Weaviate's ability to combine both significantly reduces false positives:


graphql
{
  Get {
    FinancialDocument(
      nearText: {concepts: ["loan approval"]}
      where: {
        path: ["department"]
        operator: Equal
        valueString: "compliance"
      }
    ) {
      title
      snippet
      _additional { distance }
    }
  }
}