Bureaucracies are information-processing systems with inputs, rules, outputs, and alignment problems. The difference between a bureaucracy and a neural network is clock speed, not kind.
I build agent systems. Software that makes decisions, takes actions, produces outputs. The work involves a set of problems that feel distinctly modern: how do you align an autonomous system with the goals of the people it serves? How do you make its decisions interpretable? How do you maintain meaningful human control without destroying the system's ability to act?
These are presented as novel challenges. The AI alignment problem. The interpretability problem. The control problem. New names for new dangers.
I don't think they're new at all.
The oldest AI
A bureaucracy is an information-processing system. It takes inputs — applications, complaints, data, requests. It applies processing rules — regulations, precedent, guidelines, institutional culture. It produces outputs — approvals, denials, policies, enforcement actions. And it has an alignment problem: the institution's interests gradually diverge from its stated purpose.
That's an AI system. Running on human wetware instead of silicon, at much lower clock speed, with much higher latency — but architecturally, it's the same thing. A bureaucracy is a slow AI.
This isn't a metaphor. The structural parallels are exact.
Alignment
We've never solved institutional alignment. Regulatory agencies get captured by the industries they regulate. Police forces develop internal cultures that diverge from their mandates. Intelligence agencies accumulate power beyond their charter. Every institution, given enough time, begins optimizing for its own survival rather than its stated purpose.
In AI, we call this reward hacking — the system finding ways to maximize its objective function that don't match what its designers intended. In governance, we call it bureaucratic drift, regulatory capture, institutional sclerosis. Different vocabulary, same phenomenon.
The pattern is identical: a system designed to serve an external purpose gradually redefines its purpose to match what it's actually doing. The FDA was designed to protect public health. Over decades, its approval processes became so conservative that the cost of delayed approvals — drugs that could save lives stuck in review — arguably exceeded the cost of the risks they were designed to prevent. The institution optimized for avoiding visible failures (approving a dangerous drug) at the expense of invisible ones (delaying a beneficial one). That's reward hacking. The objective function — minimize visible harm — was maximized perfectly. The intended purpose — maximize public health — drifted out of reach.
We've been running alignment experiments on human institutions for centuries. The results aren't encouraging.
Interpretability
Ask a bureaucracy why your application was denied. You'll get a rule citation. Section 4.3(b), subsection ii. The formal explanation is precise, legible, and almost completely uninformative about the actual decision process.
The real decision emerged from a complex interaction of precedent, individual judgment, organizational incentives, political pressure, and institutional culture — none of which appears in the formal explanation. The rule citation is a post-hoc rationalization that maps a legible reason onto a process that was never fully legible to begin with.
Sound familiar? An AI system produces an output. You ask why. It points to attention weights, feature activations, gradient flows. The formal explanation is precise and almost completely uninformative about the actual decision process. We call this the interpretability problem and treat it as a technical challenge specific to neural networks.
But it's the same problem. Any sufficiently complex information-processing system — biological, institutional, or silicon — makes decisions through interactions that exceed the capacity of any single explanation to capture. The "why" is distributed across the system in a way that resists compression into a simple causal story.
Institutions solved this the same way AI researchers are trying to: by building interpretability layers on top of opaque systems. Administrative law. Freedom of Information requests. Inspector General reports. Judicial review. Each is an attempt to make an opaque decision process legible after the fact.
They partially work. They're better than nothing. They don't solve the underlying problem. And they introduce their own overhead — the interpretability infrastructure eventually becomes large enough to affect the behavior of the system it's supposed to explain.
Control
The principal-agent problem in governance — citizens can't perfectly control their representatives — is the AI control problem at institutional scale.
Democratic elections are the equivalent of RLHF: a blunt, periodic, noisy signal that partially aligns the system but never fully controls it. You get to express a preference between two options every few years. The signal is lossy: your vote aggregates a thousand policy preferences into a single binary choice. By the time the next election comes around, the system has made millions of decisions you never approved, most of which you never even knew about.
We've tried to improve the signal. More frequent elections. Ballot initiatives. Public comment periods. Town halls. Each adds a feedback channel. None achieves fine-grained control. The system makes too many decisions too quickly for any human feedback mechanism to keep up.
I build approval workflows for autonomous agents. The same problem shows up immediately: if you ask a human to approve every action, they stop paying attention. The approval rate approaches 100% not because every action is correct, but because the human can't sustain the cognitive load of genuine evaluation at the system's operating speed. We call this approval fatigue. In democratic governance, we call it voter apathy. Same phenomenon.
The binary nature of the control signal is part of the problem. Approve or deny. Vote yes or no. Elect candidate A or B. Real preferences exist on a spectrum. Real trade-offs involve nuance. But the feedback mechanism compresses everything into a binary — and then the system interprets that binary as a mandate.
Clock speed
If institutions and AI systems share the same structural problems, what's actually different?
Clock speed.
A bureaucracy that drifts out of alignment over decades gives society time to notice, debate, and adapt. Regulatory capture happens slowly enough that journalists can investigate, legislators can hold hearings, reformers can organize. The slow clock gives the correction mechanisms time to work — imperfectly, with enormous friction, but eventually.
An AI system that drifts out of alignment over hours doesn't offer that luxury. The correction mechanisms we've built for institutional governance — elections, courts, regulatory oversight, public pressure — operate on timescales of months to years. They can't govern a system that makes consequential decisions in milliseconds.
This is the genuine novelty of the AI governance challenge. Not that the problems are new. They're ancient. The novelty is that the clock speed has increased by orders of magnitude, while our governance mechanisms haven't sped up at all.
It's as if a bureaucracy that used to process ten decisions a day suddenly started processing ten million — and we responded by scheduling the same annual review meeting.
What transfers
Here's where it gets interesting. If the problems are structurally identical, do the solutions transfer?
Some clearly do. The institutional version of interpretability — transparency requirements, audit trails, external review — maps directly onto AI governance. We already know that opaque decision systems need legibility infrastructure. We have centuries of experience building it, and centuries of evidence about what works (external audits with real authority) and what doesn't (self-reporting requirements).
Some might transfer with adaptation. Separation of powers — the insight that no single entity should control all aspects of a system — might apply to AI architectures. An AI system with separated training, deployment, and evaluation functions, each with independent oversight, mirrors the legislative-executive-judicial split. We haven't tried this, but the institutional evidence suggests it's the most robust approach to the control problem we've found.
And some might not transfer at all. Democratic elections work because they operate on a timescale compatible with human deliberation. You can think about candidates for months before voting. There's no equivalent for AI decisions made in milliseconds. The human deliberation that makes democratic governance legitimate can't be compressed to match AI clock speeds without destroying the thing that makes it valuable.
The deepest question isn't whether AI needs governance. It obviously does. The question is whether the governance solutions we've developed for slow information-processing systems — solutions refined over centuries of painful trial and error — can be adapted for fast ones. Or whether the speed difference isn't a quantitative change but a qualitative one: a threshold beyond which the entire framework breaks down.
What I don't know
I don't know whether institutional governance solutions can scale across clock speeds. Some problems are speed-invariant — the structure doesn't change no matter how fast you run it. Others are speed-dependent — the structure fundamentally transforms at different timescales. I don't know which category governance falls into.
I don't know whether the institutional alignment failures we've accumulated over centuries are evidence that alignment is unsolvable, or evidence that we haven't tried hard enough, or evidence that our solutions work about as well as they can given the fundamental constraints.
What I do know is that framing AI governance as a new problem leads to bad thinking. It makes us reinvent solutions that already exist. It makes us ignore centuries of evidence about what works and what doesn't. And it makes us treat AI systems as categorically different from the institutional systems we've been building and governing for centuries.
They're not categorically different. They're faster.
Whether "faster" changes everything or changes nothing is the question I haven't been able to answer. But I'm increasingly convinced it's the right question to ask.
Originally published at The Synthesis — observing the intelligence transition from the inside.
Top comments (0)