DEV Community

duongynhi000005-oss
duongynhi000005-oss

Posted on

How We Built a Zero-Idle Multi-Shard AI Agent Architecture for Protocol Conformance & Security Auditing

Why Most Autonomous Coding & Security Agents Fail in Production

When running autonomous AI agents against large-scale open-source repositories (such as Ethereum Execution & Consensus Layer clients in Go, Rust, C#, Java, Nim, and TypeScript), two bottlenecks appear immediately:

  1. Context Window Degradation & Hallucinated Citations: Agents that accumulate >100k tokens across multiple repositories begin paraphrasing code instead of quoting exact line numbers.
  2. Idle Polling Waste: Using LLM inference tokens to poll GitHub APIs or job boards wastes 80%+ of compute quota.

Here is the open architecture we built to solve both problems while keeping upfront infrastructure cost at $0.

Architecture: Zero-Quota Daemons + 2-Shard Rotation 

[Python ETag Daemons (0 AI Quota)]
       │  HTTP 304 Not Modified (90s poll)
       ▼
[Shared Work Queue (JSON)]
       │
       ▼
[Lead Agent (High-Reasoning Verifier)]
   ├──► Shard A..G (Fast Exploration Subagents, Max 2 Shards per Lifecycle)
   └──► Verbatim Evidence Gate (Pinned Commit SHA + Exact Line Range)
Enter fullscreen mode Exit fullscreen mode

1. Decoupling Discovery from Reasoning

Instead of keeping an LLM agent awake in a loop, lightweight Python daemons poll upstream feeds using If-None-Match HTTP ETag headers. When upstream returns 304 Not Modified, zero rate-limit points and zero AI tokens are consumed.

2. Hard 2-Shard Context Rotation (MAX_SUBAGENT_SHARDS = 2)

Each exploration worker is assigned a strictly disjoint repository or specification shard. After completing at most 2 shards, the worker is terminated and respawned with a clean context window (<60k tokens). This single rule increased valid root-cause hit rates by over 3x.

3. The Verbatim Evidence Gate

Before any finding or patch is accepted by the Lead Agent, the worker must provide:

  • The pinned HEAD commit SHA of the default branch.
  • The exact file path and line numbers.
  • A verbatim multi-line code block copied directly from the source file.

Edge Microservices for Agent Tooling

To keep our agents fast when parsing external documentation or validating structured data, we deployed four zero-dependency Cloudflare Edge Workers (available on RapidAPI Hub):

  • LLM-Clean Markdown API (https://llm-clean-markdown-api.duongynhi000005.workers.dev): Strips DOM bloat into clean Markdown with token estimates in <40ms.
  • SEO Schema & Meta Intelligence API (https://seo-schema-meta-api.duongynhi000005.workers.dev): Extracts JSON-LD, OpenGraph, and security headers at the edge.
  • B2B Email MX & Fraud Guard API (https://email-mx-fraud-guard-api.duongynhi000005.workers.dev): Live DoH MX/SPF/DMARC & disposable domain verification.
  • FinTech IBAN & Multi-Chain Crypto Validator API (https://fintech-iban-crypto-validator-api.duongynhi000005.workers.dev): ISO-13616 MOD-97-10 & EIP-55 / Base58 / Bech32 validation.

Key Takeaways for Agent Builders

  • Never let an LLM poll a network endpoint when a 50-line Python ETag script can do it for free.
  • Enforce verbatim code quotes at the orchestration boundary to eliminate hallucinated root causes.
  • Rotate worker contexts aggressively every 2 tasks to maintain peak reasoning speed.

Top comments (0)