DEV Community: Alexandr Zaporojan

For AI Not to Hallucinate, We Must Stop Hallucinating First: How I Built an 80-File Rule System for AI Marketing and Failed Completely

Alexandr Zaporojan — Sat, 29 Aug 2026 17:19:02 +0000

I built a 127-file deterministic SDLC framework that allows an AI coding agent to construct complex, enterprise-grade backends. Naturally, I thought: why not apply the exact same discipline to marketing? I built an 80-file rule system, created strategist and psychologist roles, ran real-world experiments—and suffered a total, spectacular failure. Here is what that crash taught me about where the actual boundaries of LLMs lie.

I disagree with the popular claim that AI cannot build software architecture. Given deterministic constraints, strict verification gates, and clear state machines, it can build remarkably robust systems.

However, I completely agree that LLMs cannot perform analysis in the genuine sense of human thinking.

In my latest experiment, I pushed that capability to its absolute breaking point—and watched it shatter.

The Origin: Building for My Wife

After my software engineering framework (the LEO Framework) proved that it could reliably prevent async deadlocks and context drift in large codebases, I had an obvious idea: why not help my wife in her work?

Through our warm life together, listening to her talk about marketing education, client campaigns, and workplace adventures over dinner and evening walks, I learned a tremendous amount about the discipline.

So, I sat down and built an engineering-grade marketing framework.

I created a Marketing Lead orchestrator. I wrote over 80 tightly coupled instruction files: dedicated roles for a Researcher, Market Strategist, Consumer Psychologist, Copywriter, Senior Editor, and a ruthless Adversarial Critic. I fed it frameworks for audience segmentation, core values, psychological triggers, and positioning maps.

The result? A total, irrecoverable collapse.

The Illusion of Strategy

The outputs looked slightly better and more structured than asking a raw prompt in a vanilla Claude chat. At first glance, to an amateur who knows nothing about marketing, the resulting strategy documents looked impressive: clean headings, bullet points, professional terminology.

To a seasoned marketing professional, however, it was a complete disaster.

The AI could execute deep information retrieval (the Researcher role was quite good at collecting facts), but it was fundamentally incapable of synthesizing that data. It would consistently latch onto a minor, spurious detail—for instance, noticing a minor inconsistency in a website's copy—and convince itself that this was the root-cause catastrophe of the entire business.

Before I knew it, the model had constructed an entire multi-month go-to-market strategy centered around fixing that one trivial discrepancy, treating it as the holy grail that would magically fix client acquisition.

The Associative Trap

Why did an 80-file rule architecture build an Enterprise IDE in Python, but produce useless corporate fluff in marketing?

The Absence of Tacit, Lived Context:
The AI does not know the unwritten nuances of the brand. It doesn't know the personal charisma, quirks, or reputations of the founders it’s trying to position. Even if you feed it a dense context file, it remains a fragile house on stilts.
The Ground Truth Asymmetry (Code vs. Psychology):
Programming is a formal, mathematical system. It is bounded by compilers, ASTs, HTTP contracts, and database transaction isolation. A PostgreSQL lock either holds or it deadlocks; a test either passes or throws an error. There are millions of deterministic patterns for the AI to rely on.
Marketing has no compiler. There is no assert customer.feels_desire() == True. Marketing is an irrational, intuitive, socio-emotional battlefield.
Statistical Weights vs. Genuine Thought:
LLMs operate on an associative plane. They are probability distributions predicting the next token based on training weights. In code, associative grammar is enough to construct working logic when constrained by rules. In marketing, statistical association simply averages the internet’s marketing text into a generic mush of corporate buzzwords. It cannot open a real creator's comment section, see who is liking the posts, analyze their profiles, understand their real-world insecurities, and extract an authentic emotional narrative.

The Crisis and the Realization

When this marketing experiment imploded, I went through a brief existential crisis: Was my software engineering framework just as broken and useless? Was I fooling myself about AI coding?

The answer was no.

Software engineering works with LLMs precisely because it is an unnatural, formal discipline. In fact, it is literally painful for the human brain to learn low-level code because our biology evolved for social intuition, not pointer arithmetic and SQL schemas. For an LLM governed by deterministic rules, code is its native physics.

Marketing, on the other hand, requires human empathy, cultural timing, and irrational intuition—the exact things evolution spent millions of years carving into the human brain.

Conclusion

All my 80 marketing rule files are still sitting on my hard drive, but I have no intention of releasing them as an open-source tool. Because they simply do not work.

To all the professional marketers out there: you can stop worrying about AI taking your jobs.

When you look past the hype, an LLM in strategic marketing is just a child playing dress-up in an adult’s suit. It has all the vocabulary, but none of the understanding.

— Alex Zaporozhan

My AI Agent Ignores My Prompts If They Break the Architecture (And Why That's a Feature)

Alexandr Zaporojan — Sat, 29 Aug 2026 09:54:42 +0000

Recently, a prominent Staff AI Engineer read my post about the LEO Framework (a 120+ file deterministic rule system for AI coding agents) and asked a brilliant, hard-hitting question:

"What happens when your system encounters a genuinely adversarial input — does the chain protocol propagate the constraint, or does the interlocking make it harder to locate the failure point?"

In other words: what happens when the human gives the AI a terrible, contradictory, or architecturally destructive prompt? Will the 120 connected files spread the disease, or stop it?

Here is my raw, honest answer straight from the trenches of a 2-month solo Enterprise build.

As the person who set specific goals for LEO, I can share from my experience what happens in practice and what blind spots LEO actually has when facing adversarial or complex input (and it definitely has them).
To start with—unfortunately, I don't have the capacity to actively develop the framework itself right now. I’d love to add a lot of things, but at this scale, it would consume all my attention, which I physically cannot afford.

Nevertheless, my experience using it continues to accumulate. Right now, LEO is guiding my solo implementation of an Enterprise IDE AI Node Constructor. 100% AI-Native. (The project is under NDA, but my contract allows me to disclose architectural decisions—this was my personal request).

I originally designed LEO with a specific goal: to help a junior developer with zero architectural background build a more-or-less working product close to an Enterprise level. Once I realized the results were actually quite solid (like medCore on my GitHub, which I intentionally open-sourced for researchers just in case I was losing my objectivity on what is "good" or "bad"—that was the first project built on early versions of LEO).
The second stage began when I saw that this result, while a bit clunky, actually held up on a project with 40,000+ lines of code (and not spaghetti code, but fairly structured, cultured code). That fired me up to make LEO bigger and stronger. That’s when the instruction library grew from 90 files to 120+.

I gave the framework a real crash test. I took a freelance gig for a very low rate (which matched the fact that I am not a seasoned engineer, not a RAG developer, not even a mid-level dev). Fortunately for the project, the clients were extremely demanding regarding the technical quality of both the processes and the result. (The goal: build a pipeline that ingests a book into RAG -> LLM text generation -> sends text to HeyGen/Sora for talking-head rendering -> delivers the final video, tests, and a live-avatar based on the lesson text and RAG library to the client dashboard).
Initially, the platform’s task was simply to generate a text lesson based on RAG. Then, as always happens, the scope creep hit. It grew from "just a lesson roughly equal to a $20 ChatGPT window output" to something much larger: a highly structural, unique lesson format.

So, "semantic search in a RAG library" morphed into "Chapter-specific search" + "Atomic chapter-level indexing" + "Batch retrieval." The node schema became: Retrieve RAG -> extract chapter outline -> assemble a lesson based on the outline, rephrasing it but perfectly retaining the structure and core principles of the original, adapted for a completely different niche/product. I considered this a real challenge.

I successfully delivered the task. The whole product took about 2 months, but in the process, I burned through four $200 Ultra Pro subscriptions + $200 on On-Demand API usage. Those who have been doing this longer can judge whether that’s a lot or a little.

The nodes have zero race conditions. (At least, the "lesson" product generates smoothly every single time. The only errors occur if the context window isn't large enough for the "extract chapter outline" node, or occasional edge cases where raw JSON leaks through unparsed).
The resulting text is highly unique to the niche. There is a specific temperature for extracting text from the outline (0.25) and a separate reasoning temperature (0.65—mostly using DeepSeek Reasoner). The text is chunked into scenes, the video rendering is broken down into multi-scenes atomically, and the product delivers a finished video.
Why am I telling you all this?

LEO can now solve genuinely complex tasks. Everything I described above was 100% AI-Native solo building in two months (plus a DB, which includes a centralized multi-tenant architecture where the Primary Organization can push/pull RAG libraries, materials, videos, and prompts to child Organizations).

I’ve built up specific experience and workflows for managing it, because you still have to drive it and know what to monitor.

The character of LEO:
First of all, if the requirements (specs) contain contradictions or weaknesses (adversarial input), LEO might misinterpret the logic. It will definitely try to build it coherently, but sometimes it fails to bring the business logic to a complete resolution (even though it has specific roles like CREATOR and BIZ for logic).

However:
LEO will immediately flag if something is architecturally incompatible. It will explicitly tell you how not to do something if you ask, and explain why. The LEAD role has a mandate for "Professional Pushback" and is obligated to voice it.

Very often it says: "I built X and Y, but I suggest refining A and B if you want. But this is an architectural decision. If you want, I can draft the ADR files, ARCH SPINE, and LEAD DECISION."
Quite often, treating me as a coding novice, it tells me: "This solution contradicts ADR X and Y. I do not recommend implementing this." If I insist, it replies: "Professional Pushback: I still did not execute this. If you really want this solution, I offer these alternatives, and I recommend picking option A (a cheap task closure) or option B (an expensive closure with architectural refactoring)."
LEO documents its decisions. While many might consider this "AI slop," this is exactly how it remembers what, where, and when it decided something. At the end of a project, it can issue a memorandum on exactly why it made a specific decision and why the requested alternative contradicts the entire system.

Often, to the point of my own irritation, I would demand it override the rules and execute a command without even reading its pushback (e.g., when I desperately needed a quick feature to show a client). It would do anything but what I asked. It genuinely only executes code if there is an approved LEAD DECISION -> ADR -> SPINE -> BATCH PROMPT for the Cursor queue. If this chain isn't approved, and you ask it to do something destructive in a single prompt, the logic goes like this: a) If it's a solo prompt -> pushback; b) If the request is buried among 7 other tasks -> it ignores it or writes in small print ("I didn't do this until you answer these questions," or outright ignores it assuming "maybe he won't notice I skipped it").

I haven't integrated the batch creation system into the main structure yet—I just ask it to do it by explaining how. I simply haven't had time. But it's incredibly convenient: a task list of 40 prompts, Cursor Queue, and a couple of meta-prompts between tasks like "If finished -> move on; if not -> refine prompt," plus "Do an audit of the completed work (mandatory)" + a global audit on the finished batch.

If any of this is helpful, I’m glad. Thank you for your time and attention—it was a real pleasure to get a response from you. It would be even better if you find some useful concepts in how the LEO framework is organized, either for yourself or for the broader developer community. Best regards, Alex.

P.S. LEO is excellent at reading logs and actually prefers working strictly from them. It spots egregious violations very easily. Medium ones take a few prompts. Minor omissions trigger a snowball of fixes from the constantly looping "do an audit of the last prompt" which inadvertently touches everything around it. Yes, sometimes it catches critical things completely outside the direct audit scope, because it might miss them during a targeted check.

Note: I mostly work on Cursor using Grok 4.5/4.6, but Claude Sonnet 5 / Opus are the ones that "wear the LEO suit" best. ChatGPT 5.5 can be magnificent, but it inflates reasoning tokens so heavily that costs go to space (though the solutions are beautiful).
Also, regarding the frontend—there are issues. It makes many things quite nicely, but the frontend still suffers unless you write all the specs immediately using Opus or GPT 5.5. The problem is I’ve mostly studied backend, so I don't fully understand how to tune the frontend rules beyond "just a good classic Mantine UI."

P.S. 2. The best part about the documentation LEO "bloats" the project with: besides fighting hallucinations, at the end of a project (especially if you keep the logs), you can describe the problems you faced and how you solved them. Then, using Sonnet 5 or ChatGPT 5.5 outside the analyzed project (otherwise the universal /roles instructions will bleed into local project specifics), you can ask it to refine the roles and passports based on those problems + whatever else it finds itself, to prevent them in the future. I think there is massive room for evolution here.

Final thoughts:
If you want to see how the LEAD role handles "Professional Pushback" or how the MODEL BLOCKER works, the entire LEO Framework is open-source on GitHub:
👉 https://github.com/alex-zaporozhan/leo

You can also read the post-mortem of how this framework prevented async deadlocks in my previous article here: https://dev.to/alex-zaporozhan/i-spent-200-on-one-queue-bug-then-i-built-a-12-node-ai-pipeline-constructor-and-forgot-the-bug-2a0m

I Spent $200 on One Queue Bug. Then I Built a 12-Node AI Pipeline Constructor and Forgot the Bug Ever Existed.

Alexandr Zaporojan — Thu, 27 Aug 2026 12:01:10 +0000

How a $200 debugging session on a hung Celery ingest queue turned into two permanent rule files — and why I didn't notice, for months, that the class of bug they were written to prevent had simply stopped happening.

The queue that only died on the cheap hardware

Here's the setup, stripped of anything that identifies the client: a RAG ingestion pipeline for an AI content platform. Celery workers at concurrency=2, pulling documents, chunking them, batching the chunks to an external embedding API rate-limited to 8 requests per second through a Valkey-backed limiter, then upserting the vectors into pgvector. Nothing exotic. The kind of pipeline every "add RAG to your SaaS" tutorial waves through in one paragraph.

On the 4 vCPU dev box, it worked. Every time. On the 2 vCPU staging profile — the one that actually matched the client's budget tier — it hung. Not crashed. Hung. The first ingest job would climb to embed_cursor=64/190 and stop. The second job would go silent immediately. The maintenance worker's own dashboard kept reporting global_active=2 — two jobs "actively processing" — while the reclaim sweep logged reclaimed:0, over and over, because as far as the system was concerned, nothing was dead. Both jobs were still sending a heartbeat. The queue was full, healthy-looking, and completely incapable of finishing anything.

I want to be honest about what this actually cost me, because "async is hard" is a sentence people nod at and then don't budget for: about $200 in agent-hours before I stopped treating it as a flaky-infra problem and started treating it as a design problem. The first four or five passes were exactly the debugging you'd expect — restart the worker, bump the concurrency, add a retry, blame Docker's CPU throttling, blame the embedding provider's rate limiting, blame the reclaim sweep's polling interval. Every one of those "fixes" made the symptom move to a slightly different spot in the log without touching the actual defect. That's the expensive kind of bug: not the one that's hard to find, the one that's easy to appear to fix five separate times.

What was actually broken (there were nine things)

When I finally forced myself to write the failure down as a state machine instead of a stack trace, it stopped being one bug and turned into a small catalogue:

The heartbeat renewal had no relationship to actual progress. A worker stuck on a hung external await kept renewing its own lease forever — the pulse looked alive because nothing was checking whether the cursor was still moving.
The rate limiter's own bookkeeping — check → increment → maybe reject as three separate steps instead of one atomic operation — consumed a token even on a rejection. Under concurrency, the bucket found ways to report itself full while doing almost no real work.
Retries after a 429 went around the shared limiter entirely, so two workers retrying at once could burst well past the declared 8 RPS ceiling, which produced more 429s, which produced more retries.
A broad except Exception around the provider call collapsed a retryable timeout into a terminal failure — so the "self-healing" retry logic was, in a meaningful number of cases, silently dead on arrival.
Retry existed on two layers at once (the Celery task's own autoretry_for and a manual service-level requeue), so a single failure could multiply into duplicate attempts that bypassed the limiter a second time.
Liveness was determined by three different, disagreeing sources of truth: a 120-second lease set at dispatch, a 5-minute "staleness" window checked by the reclaim sweep, and a updated_at timestamp that got refreshed by the very heartbeat that was lying about progress — which meant an expired lease could get "rejuvenated" back to alive.
The reclaim sweep itself ran once every 300 seconds against a 120-second lease — so a genuinely dead worker could sit holding a slot for up to five minutes before anything even looked at it.
The batch that was supposed to run in parallel under the 8 RPS budget was actually running sequentially — so real throughput was closer to 1 RPS, and the "rate limit" was never the bottleneck; unused parallelism was.
And, almost funniest in hindsight: the regression test suite had a test that asserted the old, wrong behavior — "a job stays active even past an expired lease" — so the fix, when I found it, technically broke a passing test. The test had quietly become the spec for the bug.

None of these, individually, reads as dramatic. Together, on a 2 vCPU box where there was zero slack to hide the timing, they added up to a queue that looked alive and was completely dead.

The two files that replaced a note to self I'd never reopen

I run an autonomous coding agent under a written rule system I've been building for a while — I call it LEO, and it's open on GitHub if you want the whole thing. The instinct after a bug like this is usually "note it in the ticket, move on." I did something slower and, it turned out, considerably more valuable: I turned the nine defects above into two permanent files the agent is required to read before it's allowed to touch a queue or a pipeline again.

The first file is the why — a canon that breaks the incident down cause by cause and turns each cause into a numbered law: heartbeat has to renew alongside actual progress, not on its own; the rate limiter has to be an atomic check-and-consume sitting at the wire, in front of every attempt including retries, and a rejection is never allowed to spend capacity; there is exactly one owner for each error class's retry logic, and every other layer that could also retry it is explicitly turned off; liveness has one single source of truth — a lease-expiry timestamp — read by exactly one function everywhere it matters, with the reclaim period mathematically forced to be at most half the lease TTL.

The second file is the reflex — not the theory, the muscle memory. It's a literal grep checklist: patterns like await (client|http|session).\. without a nearby timeout=, or except Exception: wrapped around a provider call, or a retry/backoff block sitting outside a shared limiter instead of inside it. The agent runs this over its own diff before calling anything done. It's deliberately mechanical — the point isn't "think about async safety," it's "if this exact text pattern shows up in your change, stop and answer this exact question before you ship."

Both files went in, along with an update to the architecture rules requiring a filled-in "pipeline passport" — retry ownership, limiter placement, lease numbers, a stuck-progress deadline — before anyone, agent or human, is allowed to start a new background pipeline. Then I moved on to the next client deliverable and didn't think about it again for months.

Then came something with a lot more surface area for this exact bug

The next chunk of that engagement was a visual pipeline constructor for the same platform — the kind of node-graph builder people compare to n8n or Zapier, except purpose-built for AI content generation instead of generic SaaS glue. Twelve node types on the canvas: a retrieval node doing hybrid vector + full-text search against the knowledge base, an LLM node with prompt versioning and a pinned-vs-latest toggle, a transform node for post-processing model output, a conditional branch node, a human-in-the-loop approval gate, an external video-generation node, an image-generation node, a voice-assignment node, a generic webhook node, a persistence node, and a scoring node for rubric-based evaluation. Users draw a graph, publish a version, and a graph-executor built on LangGraph — running on top of the same Celery workers — walks the nodes in order, writing a dual journal as it goes: relational step-by-step logs for operators, plus a full graph-state checkpoint for crash recovery.

That is, structurally, a much bigger and much more inviting target for exactly the class of bug I'd just paid $200 to understand. More external calls. More places a heartbeat could lie. More opportunities for two nodes' retries to collide on a shared provider. A human-approval gate that can sit "pending" for hours by design, which is precisely the kind of long-lived state that makes a naive liveness check look broken even when it's working correctly.

That build shipped twelve node executors, a credential-resolution layer with its own provider dispatch, a wallet-preflight cost estimate before triggering paid video generation, a separate leased-and-reclaimed job queue for the image/video side with its own queue-depth alarm — real production async surface, staged, load-tested. And at no point during any of it did we hit the deadlock shape from the ingest incident. Not once did a worker's heartbeat lie about progress. Not once did a retry storm past a shared limiter. I watched the agent stop itself mid-diff once or twice, on a grep hit neither of us went looking for — the reflex file doing exactly the job it was written for, quietly, without either of us noticing it was doing anything at all until I went looking afterward.

The moment I actually noticed

It wasn't until I sat down to write a retrospective on the whole engagement — the kind of "what did we learn" doc nobody enjoys writing — that it landed. We'd shipped something with roughly an order of magnitude more asynchronous surface area than the pipeline that had cost me $200 to understand, and the specific failure mode that pipeline taught me about had simply not recurred. Not "recurred and got caught in review." Not "recurred and cost less time to fix." Absent. I had to go check the two files' own dates against my git history to convince myself I wasn't misremembering — they were written the week of that first incident, months before a single node executor for the constructor existed.

That's a strange thing to notice about your own work, because it's the opposite of how debugging normally feels. Normally you fix a bug, and six weeks later a cousin of that bug shows up in a different file, because the actual lesson — "a heartbeat proves a pulse, not progress" — lived in your head, or in a chat log nobody re-reads, and your head is a lossy, non-versioned medium that a fresh problem doesn't automatically consult. The fix generalized this time because it wasn't stored as a memory. It was stored as a gate the agent has to pass through, the same way a database constraint doesn't care whether the person writing the INSERT remembers why double-booking is bad — the schema itself won't allow it either way.

Why this is the actual point, not a nice side effect

I don't think the interesting story here is "I fixed an async bug." Async bugs get fixed constantly; that's not news. The interesting story is that the fix stopped being something I had to remember to apply. It became infrastructure I stopped consciously thinking about — which, if you've ever tried to enforce a coding standard across a team by asking people to "please remember" it, is exactly the property you actually want and almost never get. A rule that requires ongoing vigilance decays. A rule the agent physically cannot get past — because it's a grep pattern with a mandatory stop-question attached, not a paragraph of advice — doesn't decay, because it was never running on anyone's memory in the first place.

I'll add one more thing, because it's the part that convinced me this generalizes past my own project: I periodically dump the whole rule system into a fresh, completely unprimed model session — no history, no accumulated context, just the raw files — specifically to get an adversarial read on my own constitution. The read I got on these two files, unprompted, was that the grep-based reflex works because it forces a specific kind of attention collision: the moment the pattern await client.post( shows up in a diff with no timeout= anywhere near it, the model has to reconcile that literal text with a literal rule sitting three lines below it in the same file. There's no interpretive gap left for the model to quietly skip past. That's a fairly mechanical description of why a text file changed a production outcome, and it matches what I watched happen across an entire node-based pipeline constructor that never once reproduced the bug it was written to prevent.

If you're fighting the same class of problem — a queue that works until it doesn't, on hardware that's just constrained enough to expose the race your dev box hides — the two files are ASYNC_WORKERS_CANON.md and ASYNC_AWAIT_REFLEX.md in the LEO repository. But I want to be precise about what actually made them hold, because it isn't the two files sitting alone in a folder — that's a memo, and memos get skimmed. @LEAD is what routes a queue task to those two files instead of leaving them undiscovered next to 125 others about something else entirely. @DEV is the role contractually bound to run the reflex grep over its own diff before calling anything finished, not a role that's merely encouraged to. @QA_ARCH is what checks, on every pass, that the numbered laws in the canon actually show up in the code and not just get cited in a commit message. @ARCH is what refuses to let a new pipeline start at all without the passport those two files require filled in — retry ownership, limiter placement, lease numbers, named out loud before a single executor exists. Hand the same two files to a model with no routing, no enforcement, and no design-time gate around them, and you get a well-written document nobody was ever required to open. The two files are the specific lesson from my $200. The other 125 are the reason that lesson is binding instead of optional — which is the actual claim underneath this whole piece: the reliability came from the composition, not from any one file, including these two.

P.S. A few facts change how you should weigh everything above, so they belong here instead of staying unsaid: I've been working in tech for under a year, I work alone, and I'm exactly the kind of "AI-native" developer a certain type of senior engineer treats as a punchline rather than a job description. No team standup to catch what I miss. No tech lead to overrule a bad call before it ships. No corporate process quietly doing half the discipline for me in the background. I didn't write the LangGraph executor by hand, and I wasn't the one stepping through Celery's internals at 2am. My agent was — reading the two files above, the same way ASYNC_AWAIT_REFLEX.md itself says an agent, "or, honestly, a human," is required to. What I actually did was spend the $200 turning one incident into a rule the agent can't get around, then spend the months after that directing, not typing.

I'm not saying this to lower the bar on what you just read. I'm saying it because it's the stronger version of the point, not a weaker one. "An experienced engineer solved a hard concurrency bug" mostly proves that the engineer was experienced — it doesn't tell you much about the process. What actually held the line here was a file the agent was structurally required to obey, not a decade of calibrated instinct sitting behind the keyboard, and not a team of reviewers backing it up either, because there was neither. Someone a year into this industry, working solo, with no one else in a position to catch what the agent got wrong, is a harder test of the claim than a ten-year veteran with a team behind them running the same setup would have been. It held anyway. That's the part of my background worth mentioning here — not as a disclaimer, and not as a rebuttal to anyone who'd rather dismiss the label than read the two files — but as the stress test.

— Alex Zaporozhan

Your AI Coding Agent Doesn't Have a Junior-Developer Problem. It Has an Amnesia Problem.

Alexandr Zaporojan — Tue, 25 Aug 2026 09:32:45 +0000

How 41 codified laws, 22 specialist roles, and a file-based memory system stopped an autonomous coding agent from quietly re-breaking the same production defect every few weeks — and why I'm open-sourcing the whole thing as LEO.

Ten times faster, ten times more garbage

Developers reach for Cursor and Copilot to write code ten times faster, and the tools deliver on exactly that promise — which turns out to be most of the problem. Used as advanced autocomplete, an LLM doesn't produce ten times more good code. It produces legacy at ten times the usual rate. You ask for a feature; the model hands back a wall of if/else; you ship it. Two months later the codebase reads like it was assembled by five people who never spoke to each other, the test suite is red more often than green, and the senior engineers who never touched the tool get to point at the wreckage and say, "See? AI is just a toy."

They are not wrong about the wreckage. They are wrong about what caused it.

The bug that wasn't a bug

Directing an AI coding agent on real, paying engagements — multi-tenant SaaS platforms, one of them with background AI pipelines — surfaced the same shape of defect more than once, in different files, weeks apart. My own project's changelog (roles/SYSTEM_UPGRADE_MANIFEST.md — every rule this system has ever added is logged there, with a reason) documents the pattern directly: a rate limiter that could be starved by its own retries because the check-and-consume wasn't atomic at the point of the call. A background worker whose heartbeat proved it was pinging, not that it was making progress — a zombie that looked alive on the dashboard. A held database transaction that outlived the request that opened it and sat there as a lock-holding corpse until something else timed out behind it.

Each time, the agent's code was syntactically perfect. Each time, it passed its own tests. None of this was "the AI is bad at coding" — a frontier model in 2026 writes fine syntax all day. What the log shows is something else: the fix, when it happened, didn't generalize. A defect gets patched in the file where it was noticed; the class of defect — "a heartbeat is not proof of progress," "an if check is not a lock" — has no durable home unless something outside the conversation writes it down as a standing rule. A chat window that scrolls out of context has no mechanism to carry a lesson forward into the next file, the next session, or the next agent entirely. That's not incompetence — it's closer to amnesia at the architecture level, and it is a far more expensive defect than any single wrong line of code, because it recurs on a schedule instead of getting fixed once.

That's the point at which "prompt the AI better" stopped being a credible answer and started looking like an engineering-process problem — the same kind of problem code review, QA, and architecture sign-off were invented to solve, decades before anyone typed "you are a helpful senior engineer" into a text box.

It doesn't need supervision. It needs a form to pour into.

The standard advice for this problem is "treat the AI like a junior developer" — review everything, don't let it merge unsupervised, keep a human in the loop. That's better than nothing. It's also the wrong model, because it assumes the AI already has judgment that mostly just needs correcting on the margins.

It doesn't have judgment to correct. What it has is a high-volume stream of pattern-matched output with no default shape of its own. Pour that stream into an empty context window and it takes whatever shape the last few messages happen to suggest — which is exactly why it drifts. A junior developer is calibrating an internal model of correctness that gets sharper over time. An unconstrained agent has no such model to calibrate; every session starts from the same blank slate, and "use your judgment" is not an instruction it can actually follow, because there's no accumulated judgment there to use.

So the fix isn't heavier supervision of a decision-maker that doesn't fully exist yet. It's building the banks before the water starts flowing: narrow, explicit, interlinked protocols that live inside the project itself — not a paragraph of "best practices" floating somewhere in the model's training data, and not on some vendor's central server you have no visibility into, but files in the actual repository the agent is working in. The agent doesn't need better judgment about which approach fits a given situation. It needs the approach already decided, written down, and sitting in a file it is required to read before it acts — so it isn't guessing by accident which rule applies. It's following the one that was already there.

That's the actual shift LEO makes: not "supervise the junior dev harder," but stop expecting a finished decision-maker and start engineering the channel its decisions are required to flow through.

Why "just write better prompts" doesn't scale

A single well-crafted prompt is a snapshot of good judgment. It captures what you know to ask for today. It says nothing about the bug you haven't hit yet, the invariant nobody wrote down, or the decision the agent made forty messages ago that it is about to silently re-make, differently, right now.

This is context drift, and it's the actual bottleneck of autonomous coding — not model capability. Give a model enough tokens and it will confidently:

Skip the empty state, the 500th test, the timeout on the outgoing HTTP call — because nothing in the conversation made skipping it expensive.
Report "should work now" or "practically done" with the same tone it uses for a fact it actually checked. Confidence and correctness are not the same signal, and an unconstrained agent has no mechanism to distinguish them out loud.
Never get told no. A single-agent chat has no adversary, no separate reviewer, no QA pass with different incentives than the one that wrote the code. A logic hole ships exactly as fast as the happy path does, because nothing in the loop is structurally positioned to catch it.

None of this is a hypothetical risk. It is the default behavior of an agent operating without a written, load-bearing process — which is precisely the situation almost every "AI pair programmer" setup puts the model in today: one soft system prompt, infinite goodwill, zero enforcement.

Tokens are not the budget. Organization is.

Most people trying to control the cost of agentic coding optimize the wrong variable. They shrink each individual prompt and count tokens per request, treating a smaller ask as a cheaper one. That's measuring the wrong unit. A hundred disorganized double-check loops spread across five million tokens produce a worse result than two or three tightly organized ones spread across that same five million tokens. Same spend, radically different outcome — because the cost was never the tokens themselves. It was the absence of a structure telling each loop what it was actually supposed to verify, against what written standard, and who signs off when it's done. Shrink the prompt and you still pay for a hundred aimless loops. Give the loop a gate and a criterion instead, and it often takes two or three.

What I actually built

I come from an unusual angle for this problem. Before I wrote a line of production code, I spent five years in Emergency ICU. That environment teaches you something that transfers directly and uncomfortably well to shipping software with an autonomous agent: zero-error tolerance is not a slogan, it's a set of protocols that exist because someone already paid for the version without them. You don't re-derive the correct response to a cardiac arrest from first principles under pressure — you follow a checklist that encodes every prior failure, because "use your best judgment" is precisely the instruction that fails under load.

I'll be straightforward about where this actually started, because the honesty matters more than the polish. I came into software development as a junior in the middle of the hiring freeze that's reshaped this market over the last few years, and the catch-22 inside it has a specific, brutal shape: you can't get hired without production experience, and you can't get production experience without being hired. LeetCode katas and toy CRUD projects don't close that gap — they were never built to carry the weight of a real system with real users and real consequences for getting an invariant wrong. I felt that wall directly, and it wasn't only a financial problem. It was a quiet, ongoing doubt about whether the path you actually want is still open to you at all, when every door that would let you prove yourself asks for the proof first.

So the bet underneath LEO was never really a career strategy. It was closer to: if the industry won't hand a junior the reps, and a coding agent can write code but has no engineering department to write it inside of, then build the department myself and let the agent supply the reps the market wouldn't. Production-grade SDLC discipline cannot be learned from LeetCode, and it cannot be prompted into existence one clever system message at a time. It has to be architected — as a real, versioned, falsifiable rule system that an agent is required to operate inside of, the same way a hospital shift operates inside protocols nobody re-litigates every time.

That system is LEO — Lead Engineering Orchestrator.

LEO is not a library, a plugin, or a hosted product. It's a written constitution — a .cursorrules file plus a 127-file, ~254,000-word role library — that any coding agent with file-system/tool access can load instead of a one-line personality. It gives the agent:

A single entry point, @LEAD, that routes every request to a named specialist instead of one model trying to simultaneously be the architect, the developer, and the auditor of its own work in the same breath.
Twenty-two roles with real jurisdiction — @ARCH owns the database and API contracts; @PRINCIPLE owns whether a state is logically reachable at all, independent of whether it's technically buildable; @QA_ARCH audits business logic before anything ships; @PENTEST holds a genuinely blocking security verdict, not an advisory checklist; @DEV is the only role permitted to touch code, and it is allowed to refuse — to raise a MODEL BLOCKER — rather than guess at a hole in the spec.
Forty-one Absolute Laws, several written the week a specific defect happened, not from a whiteboard session — the project's own changelog, roles/SYSTEM_UPGRADE_MANIFEST.md, logs them with dates. The rate-limiter-and-heartbeat incident above (v6.25) became a law about atomic check-and-consume at the point of the call — not a comment, a permanent, greppable rule that a @QA_ARCH audit now checks for on every future pipeline — and a redefinition of "heartbeat" itself: a pulse without progress is not aliveness. The lock-holding-corpse transaction became Law 35: every session in the architecture spine now carries a numeric idle_in_transaction_session_timeout, not a hope. These aren't abstract best practices; they're scar tissue, converted into text the agent cannot argue its way around because it's not a suggestion — it's the operating rule it's loaded under.
A gate protocol, not a pipeline. The difference matters enormously: a pipeline lets an agent "complete" a phase by declaring it complete. A gate requires a written artifact — an architecture spine, a security threat model, a QA report — that a separate pass checks against a concrete, numeric criterion. roles/LEAD_ANTI_CHECKBOX_PROTOCOL.md exists specifically to catch phrases like "most likely implemented" and force a real answer: verified, with evidence, or explicitly unknown.
A memory that survives the context window. This is the actual fix for the amnesia I started with. Every non-trivial decision — the tenancy model, the threat model, the QA verdict — gets written to a file, not left in chat history. Three weeks and forty conversations later, a completely fresh agent session with zero memory of any prior chat reads that file and inherits the decision exactly. It doesn't re-derive it, and it doesn't quietly re-decide it differently. State, not history — that single principle is doing more work against context drift than any prompting trick I've found. Concretely, this means treating your IDE's own indexing of the project — Cursor's index, a plain file read, a grep across the repo — as the durable memory, and the chat window as what it always was: disposable scratch space. Of every reliability trick I tried, moving memory out of the window and into the file system is the one that actually held up once the project got large.
A human hand on the only lever that matters. LEO can evolve its own rules after an incident — but only via an explicit @EVOLVE command from a person, never automatically. And separately, Law 40 — the one I'd call the most important line in the whole system — states plainly that the agent never runs git commit, git push, or git merge, under any framing, even if you paste the exact commands and ask twice. An autonomous agent that can silently rewrite its own rules and publish its own work to history is not a tool anymore; it's an unaccountable process. LEO deliberately keeps a human's hand on both levers.

Why LEO doesn't divide and conquer

The obvious way to control both cost and hallucination in a multi-role agent system is isolation: give each role the minimum context it needs, wall roles off from each other, and assume a smaller context means fewer wrong guesses. I built LEO the opposite way, after watching isolation produce more invented behavior, not less.

The reason is mechanical, not philosophical. A role that has no idea what the adjacent role does, checks for, or expects has to fill that gap somehow — and it fills it by guessing, which is just hallucination wearing a job title. The right hand that doesn't know what the left hand is doing doesn't stay idle out of caution; it invents a theory about the left hand and acts on that theory. Isolate roles to save tokens and produce "minimal fantasy," and what you actually get is every role quietly inventing its own model of every other role's job — because nobody told it the real one, and it can't operate on nothing.

LEO's 22 roles are not isolated specialists reading their own private slice of instructions. They're facets of one written system, and every role's constitution names every other role's obligations, not just its own. @DESIGN knows a @QA_VISUAL exists that will render its spec and measure it against hostile content at four viewport widths — and it writes a more disciplined spec because that check exists, the same way a surgeon documents a procedure more carefully knowing the next shift reads the chart, not because anyone is standing over their shoulder in the moment. That mutual awareness isn't redundant overhead. It's the actual mechanism that keeps 22 roles from drifting apart into 22 private guesses about what "done" even means.

So LEO's shape isn't "divide and conquer, keep every role's window small." It's one constitution that every role operates inside of, where each role's job is legible to every other role — because the alternative to a role knowing the whole system isn't a cheaper, smaller role. It's a role silently inventing the missing half of the system on its own, every single time it hits a gap nobody told it about.

Naive multi-agent systems multiply entropy. Constitutional multi-agent systems decrease it — because mutually aware roles create a continuous gravitational pull toward quality.

This isn't a thought experiment

I've directed over 14 billion tokens of iterative, gated agentic work through this system across real client engagements and independent products — not toy demos. MedCore, a multi-tenant dental-clinic operations SaaS with row-level security, advisory-lock concurrency protection, and 800+ tests. An AI training platform with executable LangGraph agent graphs, pgvector-backed retrieval, and a generative-media pipeline. A public-sector education platform where SSR-for-SEO and license-compliance weren't afterthoughts because Law 29 and Law 41 made them non-negotiable before the first line of layout code existed.

Every one of these systems stress-tested a different part of the rulebook. MedCore is why the data-integrity canon insists that "no double-booking" lives in a database constraint, never an if statement — because an if check is exactly the thing that fails when two requests race, which is precisely when the invariant matters. The AI platform is why there's a separate @AI_ENGINEER lane instead of folding retrieval and agent-graph decisions into generic backend architecture — RAG has its own failure modes (stale embeddings, duplicate generation on retry) that a standard code review doesn't have a vocabulary for. The public site is why a search-visibility decision (SSR vs. SPA) gets made at the architecture-spine stage, not discovered as a regret after launch.

I'll say plainly where MedCore sits relative to the other two: it was my own first attempt at directing this entire process end-to-end, and it shows — it's rougher in places than the engagements that came after it, closer to a "no-code" debut than a polished flagship. I'm publishing it with those flaws intact rather than hiding them, because a debut that shows the process holding up against large, genuinely messy data going into a build — instead of a curated demo that hides how the process actually went — is the more honest evidence, and the more useful one. It'll be public at github.com/alex-zaporozhan/medCore.

Is this "open source"? Be precise about it.

I want people to use LEO freely — including at work, including as the backbone of software they charge money for. I do not want someone to take these 127 files, reskin the headers, and sell it as a competing framework. Those two goals sound compatible with "open source" until you actually read the Open Source Definition's field-of-use clause, which explicitly forbids exactly the second restriction. So I'm not calling it that.

LEO ships under PolyForm Shield 1.0.0 — a source-available license with a noncompete clause instead of a noncommercial one. Practically: use it for anything, including your own paid product. Just don't resell LEO itself. It's the same license I already ship on MedCore, for the same reason, and I've written out the full comparison against MIT, GPL, and the Creative Commons NonCommercial family in LICENSING.md — because I'd rather be boringly precise about this than technically inaccurate and hand a critic an easy, correct point to make.

What I'd want you to take from this

If you're building with autonomous coding agents and you keep hitting the same category of defect in a different file — that's not a model problem you fix by switching to a bigger model. It's a process gap, and the fix looks exactly like the fix for the equivalent gap in human teams: named ownership, written contracts between roles, an adversarial check that isn't the same pass that wrote the code, and a memory that outlives the conversation.

I'll say the underlying motive directly, because I think it matters more than the rule count. I don't want "AI just writes garbage" to become the settled opinion of the software industry, and I think it's currently on track to become exactly that — for reasons that have everything to do with process and nothing to do with the technology's actual ceiling. I think agentic coding is a genuine inflection point, not a toy, and it's already changing how software gets built whether any of us are fully ready for that or not. Publishing LEO is my attempt to push that outcome in a specific direction: make AI-driven software development measurably more stable and reliable at a global scale, put a concrete idea in front of people who are hitting the same wall I hit, and let the world decide if it's useful. That's the whole bet.

And if you're a junior stuck in the same catch-22 I described above — locked out of the experience you need because nobody will hand it to you first — I'd like this to land for you specifically, not only for teams already inside the industry. Underneath the 41 laws and the 22 roles, LEO is one junior who knew the fundamentals and nothing else, trying to organize an AI's work through instructions precise enough that the AI could open a door the market had closed. If any part of this helps you turn a tool people keep insisting only writes garbage into the engineering department willing to let you become whoever you're actually trying to become — do it, even when the conditions in front of you say it isn't possible. Mine said exactly that, for a long time, before they stopped.

That's what LEO is. It's a constitution, not a product — text you can read in an afternoon, delete the parts you don't need from, and hand to whatever agent you're already using. The repository, the full role library, the architecture deep-dive, and the real shipped systems it ran are all here:

github.com/alex-zaporozhan/leo

I'm open to Founding Engineer roles, AI-native full-stack positions, and strategic AI-SDLC architecture contracts. If any of this resonates, I'd genuinely like to hear from you — LinkedIn.

— Alexandr Zaporozhan