I want to start with a moment most of us have lived through.
It's 3 a.m. A dashboard is red. You're eight terminals deep in grep, trying to work out which service actually fell over and why. And the whole time there's this nagging feeling that you're doing archaeology on a system you wrote last month.
Here's what got under my skin about it. The answer was never actually lost. Back in the source code it said, in plain terms, that the payment service talks to Postgres through a connection pool. That this retry backs off three times. That this particular dependency is external and you must never, ever try to "just restart it." That was all right there at build time. Then we packaged everything up, deployed, threw that structure in the bin, and asked a sleep-deprived human to reconstruct it from log lines.
That gap bugged me enough that I spent a while building something around it. This post is about that.
Autoscaling is good at the wrong problem
We've gotten genuinely good at reacting to resource pressure. Traffic climbs, a box gets slow, CPU pins, and the autoscaler adds capacity or sheds load. No complaints there, it's kept things running for years.
The problem is it has no idea what your app is for. It can't tell a service that's slow because it's healthy and hammered from a service that's fast because it's quietly writing garbage to the database. It never had a model of the application in the first place.
So a whole category of failures just sails right past it. A connection pool getting drained by something downstream. A poison message kicking off a retry storm. A schema change that breaks one code path and leaves the other one looking perfectly fine. Infrastructure that only thinks in CPU and memory is blind to all of that.
The "throw an LLM at it" era
The going answer right now is to bolt a large language model onto your observability stack. Fire hose all the logs, traces, and metrics at a big central model and ask it what happened.
I get why. I also think it's the wrong shape, and I keep coming back to three things.
It's expensive in a way that doesn't survive a finance review. Frontier models bill per token, telemetry is enormous, and you're proposing to push all of it through the priciest thing in your stack.
It's centralized, so you've built yourself a single brain that now needs its own babysitting and happens to sit a long way from the failure it's supposed to understand.
And the part I actually find frustrating: it hallucinates architecture. We hand the model a heap of logs and ask it to infer who calls who, what a retry means here, what's safe to restart. That information wasn't fuzzy or missing. It was sitting in the code, exact and unambiguous, until we deleted it at deploy time and then paid a model to guess it back, usually worse.
Which made me stop asking "how do we read logs better" and start asking a different question: why can't the application just reason about itself?
What I ended up building
The thing I built, the Cognitive Autonomic Framework, does four things. I'll be honest that none of the individual pieces are new. The bet is on how they fit together.
The self-model comes out of the code. At build time a compiler walks the AST and pulls out what I call a Runtime Semantic Topology, or RST. It's a small, dense graph: who depends on who, failure domains, retry policies, latency budgets, and the closed list of repairs an agent is even permitted to try on each node. A whole microservice graph fits in a few kilobytes, so it just rides along inside the container. When something breaks, nothing has to guess the system's shape. It reads it.
A node in that graph is roughly this:
{
"service": "payment",
"depends_on": ["postgres", "stripe"],
"failure_domain": "billing",
"criticality": "high",
"retry_policy": { "max_attempts": 3, "backoff": "exponential" },
"external_dependencies": ["stripe"],
"permitted_repairs": ["restart_pool", "shed_load", "open_circuit"]
}
The bit I care about is permitted_repairs, and the fact that it's a closed list. The agent is allowed to restart the pool. It is not allowed to restart stripe, because the model flatly says that's an external dependency. The safety lives in the structure, not in whether the model happened to be in a good mood.
Reasoning stays near the failure. Every service gets a small local agent, and they sit in three tiers:
Tier 0 reflex circuit breakers, timeouts ~ms free
Tier 1 node-local reads evidence + self-model ~100ms cheap
Tier 2 global frontier model, off to the side rare expensive
Most incidents never leave Tier 0, where boring, dependable machinery handles them. Tier 1 reads the local evidence, checks the self-model, and tries a bounded fix. The big expensive model at Tier 2 only gets pulled in for the genuinely nasty stuff, and even then it's off the critical path, not sitting in the middle of it.
Nodes gossip beliefs, not heartbeats. Usually gossip is health and counters. Here nodes gossip hunches: "I think the billing pool is saturated, confidence 0.9, here's my evidence." If neighbors independently land on the same thing, confidence climbs and you've got a systemic fault. If nobody else sees it, it's isolated, and the node can heal it locally. Telling "my problem" apart from "everyone's problem," with no central coordinator making the call, turns out to be the useful trick.
Autonomy is fenced in on purpose. This is the part I won't wave away. An agent can never run an arbitrary command. It picks from the closed set the model declared for that node, and any repair it proposes has to clear verification in a shadow environment before it's allowed anywhere near production.
proposes -> in permitted_repairs? -> passes shadow check? -> ships to prod
| no | no
v v
reject reject
The model suggests, the pipeline decides. I wanted safety to be a property of the plumbing, not a thing you hope the model gets right.
What I can actually stand behind
There's a running implementation, not a diagram. Six real services, each in its own container, agents gossiping beliefs over actual TCP, and a diagnostic reasoner that's deterministic out of the box but swaps out for a hosted model if you want one. It's public, and every number in the paper came off that running system. Nothing's simulated.
What it showed:
- Giving the reasoner its self-model roughly doubled how often it found the real root cause.
- That came from the structure, not the model size.
gpt-4o-miniandgpt-4oscored the same. - The safety boundary threw out every repair it wasn't allowed to make, including the ones where the model confidently pointed at the wrong thing.
- Per-node overhead stayed small.
And what I'm not claiming, because it's easy to oversell this stuff: I haven't shown it holds at fleet scale, across a wide spread of fault types, over a long deployment. The economics pitch, that most incidents get handled cheaply and locally, is still something I'm measuring, not a result I get to wave around yet. The limitations section in the paper is written to be read, not skimmed past.
Why it might matter past DevOps
The framing I actually care about isn't "AI for ops." It's software that carries an explicit model of itself and reasons over it. Reliability is just the first place that's obviously useful. The same fabric could just as easily carry adaptive routing, or intrusion detection, or a service that rewrites its own model as it changes.
The whole thing boils down to one stubborn idea: an application should carry and reason over a model of itself, instead of handing that job to some outside observer that has to rebuild the picture from scratch every single time something goes wrong.
If that lands for you, or if you can see exactly where it falls apart, I'd genuinely like to hear it.
Listen: [https://notebooklm.google.com/notebook/4779c275-9060-44bb-b0a9-bd2eb953592f?authuser=6]
Paper: zenodo.org/records/21352597 (DOI 10.5281/zenodo.21352597)
Code: github.com/tflux2011/caf-prototype (archived at DOI 10.5281/zenodo.21363579)
I'm Tobi Adeosun, an independent researcher in Texas. You can reach me at me@tadeosun.com.
Top comments (0)