DEV Community: Doby Baxter

AI as a Fluency Machine: What Software Engineering Taught Me About Checking a Model's Answer

Doby Baxter — Sun, 13 Sep 2026 16:33:01 +0000

We refuse to install a dependency without a lockfile, a checksum, and a license audit. Then we paste a language model's answer straight into a model file and call it velocity. The scrutiny didn't disappear — it just stopped applying to the thing writing the code.

The paradox

We have built an extraordinary apparatus of distrust around our own work. Lockfiles pin transitive dependencies to exact versions. Checksums verify the bytes arrived intact. Signed commits prove authorship. CI refuses a merge over one failing test. Code review exists because we decided, collectively, that no engineer's confidence in their own code is sufficient evidence that the code is correct.

Then a model produces forty lines, and they go in.

Not because anyone sat down and decided generated code deserves less scrutiny. It's that every mechanism we built assumes a human author on the other end — someone who can be asked why, who hedges when unsure, who owns the thing afterwards. Generated code walks past all of it, because none of it was pointed at that door.

The standards haven't changed. The risk hasn't changed. Only one thing has: the single participant in the process that cannot be held responsible for being wrong is now also the one we check the least.

As a software engineer, that asymmetry tells me the real disagreement isn't about whether AI writes good code. It's about what we are willing to call checked.

Hallucination isn't the interesting question

Most debates about AI and code ask whether the model hallucinates. Fair enough — it does. But hallucination is the easy failure, because it announces itself. An invented API doesn't exist. The import fails, the type checker complains, the test goes red. The system you already have catches it.

The failure that matters is quieter.

During development, my 7B cited odoo.onchange-is-not-validation — a real rule, in my corpus, traceable to upstream documentation — and then wrote an @api.onchange to guard the value. Every citation verified. The answer was still wrong.

Citation existence is not rule compliance. That one sentence is what the whole project is built on.

I've come to think of the underlying machine as a fluency machine. I use the word deliberately. It doesn't manufacture wrong answers — wrong answers predate it by the entire history of the profession. What it manufactures is the appearance of having checked. A junior engineer who is unsure hedges, and you can hear it in the hedge. A model that is unsure produces the same fluent, well-structured, confidently-cited paragraph as a model that is right. The uncertainty signal we spent careers learning to read simply isn't in the output.

Fluency is not a confidence interval. It just renders like one.

A prompt is not a specification

Here's where software engineering has something to say. Our entire discipline runs on abstraction — interfaces, APIs, frameworks, platforms — each layer letting one engineer reason about one problem without drowning in all the others. A prompt is an abstraction too. It compresses everything you know about a domain into prose and hands it over as though prose were a contract.

But every abstraction hides what it discarded, and prose hides more than most, because prose has no schema. You can't diff it meaningfully, you can't validate it, and you cannot test whether a paragraph still means what it meant before you edited it.

I found the edge of that the ordinary way. Prompt tuning plateaued after two iterations. Hardening the Odoo rule about returning false for an empty field fixed the scalar cases and simultaneously broke the Many2one tuple handling that had been correct the day before. A 7B holds a bounded number of constraints; past that point, more instruction text just trades one error for another.

The same limit showed up in routing. My first attempt used a single system prompt covering both Odoo and TypeScript. Asked a plainly Odoo question — cap a sale order line quantity — it answered entirely in TypeScript with a Zod schema. The instructions bleed. A small model doesn't reliably route between two stacks described in one paragraph.

So I moved the rules out of the prompt and into data. Each rule lives in corpus/rules/*.json against a JSON Schema: a citable id, a severity, the failure it prevents, a wrong/correct code pair, an upstream documentation URL where one exists, and where possible a regex that detects the violation in generated code. The system prompts and the retrieval index became build artifacts, generated from that corpus by nexa build. Same principle as z.infer<typeof Schema> — one definition, derived artifacts, no drift.

That didn't make the model smarter. It changed the failure mode. Instead of rewriting prose and hoping, a rule is written once and flows to the prompt, the retrieval index, and the compliance checker together.

Nobody misconfigured it, but the system decayed

Distributed systems engineers know the shape of the next part. A latency bug in one service cascades through a queue, triggers retries, saturates a database, and surfaces as an outage somewhere else entirely. Every component behaved to spec. The system as a whole did something nobody designed.

I benchmarked the 7B at 37 tok/s. Four instrumented runs later, the report showed roughly 10. I offered three diagnoses; the first two were wrong.

The actual cause was that Ollama's memory estimator is conservative and leaves layers on the CPU with VRAM still free — 11.7 tok/s automatic versus 38.6 forced, same model, same context. My Modelfiles carried num_gpu 99, but the pipeline was calling the raw base model, which has no such parameter. Meanwhile the KV cache environment variables had been set while the server was already running, so they never took effect at all. And two orphaned llama-server processes were sitting there holding 1.4 GB of VRAM, left behind when their parent was killed.

Nobody misconfigured anything. Ollama's estimator was being careful. Windows was doing what it does with environment variables set after a process starts. The orphans were doing what orphans do. Collectively, the system ran at roughly a third of its speed for most of a working session — while I read a number I had measured once and believed ever since.

An unmeasured number decays into a claim.

That is the entire reason nexa doctor exists, and why it checks GPU residency and each of those traps by name rather than just confirming that Ollama is up. There's an old rule in operations: someone owns every service. Complexity raises the effort of attribution. It never removes the need for it.

Patterns inherit their author's blind spots

You can't check for "is this code correct" — the phrase is too vague to be an objective. So, exactly as we do everywhere in engineering, we check proxies. A regex matching @api.onchange in a validation context is a proxy for "this code is using the wrong mechanism to hold an invariant."

Individually, these are reasonable. The problem is that a proxy always discards information, and it quietly inherits the assumptions of whoever wrote it.

So the patterns get tested too. Every rule already carries a wrong and a correct sample, which means the test data existed for free. nexa validate fails the build on three faults: a pattern that doesn't match its own wrong sample catches nothing; a pattern that fires on its own correct sample cries wolf; a satisfied_pattern that clears its own wrong sample is inert by construction.

Then nexa crosscheck fires every pattern at every other rule's correct sample — because self-testing proves a pattern handles its own case and says nothing about whether it goes off on unrelated good code. The first run found three real defects. float-comparison fired on qty > 0, including inside SQL where float_compare doesn't apply; narrowed to equality, which is the actual trap. odoo-read-returns-array fired on any Schema.parse(await ...). And ondelete-is-a-decision correctly flagged a sample in my own corpus that didn't follow its own rule.

That last one is the one I'd keep. The checker was right and the corpus was wrong.

It's also the loop worth being nervous about: my rules become the system prompt, the system prompt shapes the answer, and the answer is then checked against the same rules that produced it. A rule that is subtly wrong is wrong in three places simultaneously, and the pipeline will report VERIFIED at every step. All models are wrong but some are useful — the danger was never the simplification. It's forgetting what got discarded in it.

Which is why the limitation goes in the docs, in plain words: 27 of 32 rules trace to upstream documentation, and the rest rest on general knowledge and deserve more scepticism. These rules were written by an assistant, not harvested from production incidents. Rules drawn from real incidents would be worth more.

"Verified" can be theatre

The standard reassurance in this space is that the tool validates its own output. From an engineering perspective, that tells you almost nothing about how the system behaves. Inserting a step into a pipeline doesn't guarantee verification, any more than a deployment approval guarantees oversight — an approval only matters if the reviewer has the information to spot risk, the authority to say no, and the time to actually look.

We have a name for the human half of this: automation bias. Once software earns a reputation for being usually right, reviewers start confirming its output instead of interrogating it.

The checker half has a sharper version of the same problem. A checker that fires on good code is worse than no checker, because it teaches you to dismiss it — and once you're dismissing it, it will be dismissed on the day it was right. So five of my rules deliberately carry no pattern at all, with the reason recorded for each. Two need set comparison between a decorator's arguments and the attributes its body reads, which is AST work, not regex work. One lives in a diff between module versions. Two depend on context a snippet doesn't contain. Those rules return no verdict rather than a false pass. 84% coverage with an honest list of the gap beats 100% with five patterns producing noise.

So "did it verify?" is the wrong question. The right one is: what exactly was checked, and what does the absence of a finding actually prove?

In Nexa, VERIFIED means the checks that exist found nothing. It does not mean correct. Compliance is only checked against rules retrieved for that question, plus pinned ones — so a rule that was never retrieved is a rule that was never enforced. I'd rather ship that sentence than a green checkmark.

Small and local is not a lesser tool

Now flip the paradox around.

A 7B-to-30B local model does not match a frontier model on hard architectural reasoning. That gap is a function of scale, and no amount of prompting closes it. I put that in the README, near the top, deliberately.

But engineers make a distinction when optimizing: removing overhead is not the same as increasing capability. Stripping out overhead lets existing resources do more — it doesn't upgrade the hardware. Nexa doesn't make a small model smarter. It removes the thing that made a small model untrustworthy: the gap between an answer and any means of checking it.

And for this particular job, that trade isn't obviously a loss. The frontier model has more capability. The local one has more provenance — it answers from rules I can read, cite, version and test, on hardware I own, with client code that never leaves the laptop. No API key, no network call, nothing phoning home. On a machine used for client work that isn't a feature, it's a precondition, which is also why telemetry records rule ids, counts, model names and timings, and not question or answer text unless you explicitly turn it on. A telemetry file is the last place client code should quietly accumulate.

Same technology, different question. One asks how capable this can be. The other asks how wrong it can be without me finding out.

Verification cannot be delegated to fluency

Every mature engineering discipline eventually learns the same lesson: a system can't be trusted merely because it functions. It has to be observable, auditable, and — this is the part we keep skipping with generated code — falsifiable. We'd never run production without logs, traces and deployment history, because once causality goes opaque, reliability becomes impossible. Yet we routinely accept generated code on terms we'd never accept from a library: no provenance, no stated assumptions, no way to ask what it was checked against.

Observability can't be bolted on after a system fails. Verifiability follows the same rule. If a tool isn't designed to make its own answers checkable from the start, no amount of downstream review will conjure that property later — you'll just be reading fluent paragraphs faster.

So if one idea survives this whole article, let it be this:

The true measure of a coding assistant is not how often it is right. It's how reliably you find out when it is wrong.

Correctness can't be prompted into existence. It can't be inferred from formatting, borrowed from a real-looking citation, or assumed because the prose was confident and the rule id resolved. Someone still has to write the rule down in a form that can be checked, decide what a failing check means, and accept what the gaps cost. Those decisions stay irreducibly human — which is precisely what I want from a tool I let anywhere near a client's codebase.

Nexa is source-available under PolyForm Noncommercial 1.0.0 — free to read, run, modify and evaluate for any noncommercial purpose. It's a few hundred kilobytes of Python (stdlib only) that orchestrates open models you pull from Ollama; no weights ship with it. 32 rules today across Odoo, TypeScript and JavaScript, 27 with an automated check, 0 cross-hits.

The ARCHITECTURE.md records every decision with the evidence behind it, including the ones that were wrong first — and an honest account of what it cannot do. If you only read one file, read the limitations section.

When Is a System Not Fit for Purpose?

Doby Baxter — Tue, 08 Sep 2026 18:27:47 +0000

The question

When do we get to say that a system is not fit for purpose?

In software engineering, we usually have a fairly straightforward answer. A system is fit for purpose if it reliably does the thing it was designed to do.

That sounds obvious, but we seem to apply a different standard to public systems.

A government service can have a department behind it, an application portal, a helpline, a set of criteria, a budget, and hundreds of people working within it. It can process thousands of applications and schedule thousands of appointments.

None of that necessarily tells us whether it is actually achieving its purpose.

A system existing is not the same thing as a system working.

And from a developer's perspective, that distinction matters.

Activity is not an outcome

Imagine an application with a perfectly functioning workflow:

User
  ↓
Application
  ↓
Validation
  ↓
Assessment
  ↓
Decision

Every component works.

The application is submitted successfully.

The validation passes.

The assessment takes place.

The decision is recorded.

From an operational perspective, the system has performed exactly as expected.

But what if the user finishes the process without receiving the support they actually needed?

The system has produced activity, but not an outcome.

This is a familiar problem in software engineering. We can build dashboards showing requests processed, jobs completed, transactions handled, and uptime maintained, but those metrics don't necessarily tell us whether the system is solving the problem it was built to solve.

Applications received are activity.

Appointments attended are activity.

Cases opened are activity.

Forms processed are activity.

The more important question is:

Did the user's situation actually change?

That is an outcome.

And when a system measures activity more effectively than it measures outcomes, it becomes possible for the system to look healthy while the people using it experience failure.

The edge-case problem

One of the first things you learn as a developer is that the happy path isn't the whole system.

You can design a beautiful workflow for:

Input → Processing → Success

and still have a terrible system if you haven't thought carefully about what happens when the input doesn't fit your assumptions.

Real systems encounter:

invalid input
missing information
unsupported configurations
unavailable dependencies
unexpected states
conflicting requirements
users who don't behave exactly as expected

Good software doesn't pretend those cases don't exist.

It detects them.

It explains them.

And, where possible, it provides a recovery path.

Public services have the same problem.

A system should be able to tell someone not only that they qualify, but also when it cannot provide what they need.

That isn't necessarily a failure.

In many cases, early rejection is better system behavior than indefinite processing.

If an API knows that it cannot fulfill a request, returning a clear error is preferable to accepting the request, passing it through ten other services, consuming resources, and eventually failing without explaining why.

The same principle applies to services involving people.

If a program cannot realistically help someone, keeping that person inside the process for months isn't evidence that the program is working.

It may simply mean the system doesn't have a meaningful failure state.

What a good failure state looks like

Consider two systems.

System A

A person enters a service.

The system determines that it cannot help them.

It explains why and provides an appropriate onward route.

Person
  ↓
Assessment
  ↓
Not suitable
  ↓
Explanation
  ↓
Alternative service

The person hasn't received the original service, but the system has still produced a useful outcome.

Now consider System B:

Person
  ↓
Application
  ↓
Appointment
  ↓
Documentation
  ↓
Follow-up
  ↓
More documentation
  ↓
More appointments
  ↓
Eventually: cannot help

System B may produce considerably more measurable activity.

But it is arguably the worse system.

From an engineering perspective, the difference is obvious.

System A has a defined failure state. System B has an expensive failure path.

The user spends time and resources discovering something the system could potentially have established at the beginning.

That's not just inefficient.

It's poor system design.

The infinite loop

There is another failure mode that becomes particularly interesting when you look at public services as systems: the infinite loop.

Suppose a program is intended to help bridge the gap between a candidate and an employer.

The intended architecture might look something like this:

Candidate
   ↓
Program
   ↓
Employer connection
   ↓
Employer engagement
   ↓
Employment

But what happens if the program can only meaningfully intervene after an employer has already engaged?

The process becomes:

Candidate applies
   ↓
Employer does not engage
   ↓
Application rejected / ignored
   ↓
No employment
   ↓
No employer-side intervention
   ↓
Candidate applies again
   ↓
Repeat

Nothing has technically crashed.

There may be no obvious software equivalent of an exception.

The system is simply unable to transition into the state required for success.

In software terms, I would immediately ask:

What state transition is missing?

If the purpose of the system is to help create an employer connection, but the system depends on an employer connection already existing before it can provide that intervention, then there is a circular dependency.

The system is waiting for the event it is supposed to help create.

That deserves architectural scrutiny.

Every system has boundaries

This is also where transparency becomes part of system design.

Every system has a boundary.

An API has supported inputs.

A database has constraints.

A service has capacity.

A platform has supported environments.

Good systems document those boundaries.

You don't want a developer discovering after six months that an API simply cannot support the operation they have been building around.

The same should be true for services involving people.

If a program only has established employer relationships in particular sectors, that matters.

If certain types of work are outside its scope, that matters.

If particular circumstances mean the service is unlikely to produce a useful outcome, that matters.

The user needs that information early enough to make decisions about their own time.

Otherwise, the system effectively hides its own limitations and leaves the user to discover them through repeated failure.

Observability matters

Modern software systems rely heavily on observability.

We want to know:

what happened,
when it happened,
where it happened,
why it happened,
how long it took,
where the failure occurred,
and what happened afterward.

Without observability, debugging becomes guesswork.

The same principle should apply to public systems.

If a program claims to connect people with employers, we should be able to answer questions such as:

Which employers have established relationships with the program?
Which sectors are covered?
How many participants were successfully connected through those relationships?
How many were simply advised to submit standard applications?
How many cases were identified as unsuitable?
What happened to those people afterward?
How many received an effective onward referral?

These aren't hostile questions.

They're essentially observability questions.

If the system cannot answer them, then it becomes difficult to determine whether the system is performing the function it was designed to perform.

Metrics can create the wrong behavior

This is another familiar problem in software engineering.

What you measure influences what you optimize for.

If an engineering team is measured primarily on the number of tickets closed, engineers will naturally become better at closing tickets.

That doesn't necessarily mean the product is improving.

If a call center is measured primarily on call duration, reducing call duration becomes an obvious optimization target.

That doesn't necessarily mean customers are getting their problems solved.

The same problem can exist in public services.

If success is primarily represented through:

applications,
appointments,
interactions,
cases,
referrals,
and completed processes,

then the organization can become extremely good at generating activity.

But activity isn't necessarily progress.

The difficult metric is the one that matters most:

What changed for the person?

Did they get employment?

Did they receive the support they needed?

Did the barrier actually get removed?

Did they reach another service that could help them?

Those are outcome metrics.

And outcomes are often considerably harder to measure than activity.

The user becomes part of the error-handling layer

There is another cost that doesn't always appear in system metrics.

When a system fails, somebody has to compensate for the failure.

In software, that might mean a retry mechanism, a dead-letter queue, a fallback service, or an operator intervention.

In poorly designed human-facing systems, that responsibility can quietly move onto the user.

A form fails, so they try again.

An email goes unanswered, so they follow up.

A verbal commitment isn't recorded, so they document it themselves.

A case is transferred, so they reconstruct the history.

A process doesn't explain what is happening, so they maintain their own timeline.

Eventually, the person becomes the system's external state store.

They keep the records.

They track the state.

They reproduce missing context.

They provide the observability the system itself failed to provide.

From a developer's perspective, that's a serious design smell.

The system has outsourced state management to the user.

Documentation becomes infrastructure

This is why documenting every interaction can become so important.

Emails, dates, appointments, names, decisions, promises, and supporting documents effectively become a second system running alongside the official one.

The user maintains it because the official system may not reliably preserve the context they need.

Imagine having to manually maintain your own audit log because a production system doesn't provide one.

You would consider that an architectural problem.

Yet people navigating complicated administrative systems can end up doing exactly this.

They become responsible for preserving the state of their own case.

That work has a cost.

It takes time.

It takes attention.

It takes organization.

It takes emotional energy.

And none of those costs necessarily appear in the institution's metrics.

Accessibility is part of system design

This becomes even more important when the system is intended to support disabled people.

A system that requires sustained attention, precise documentation, repeated communication, and extensive administrative work may unintentionally create barriers for the very people it is intended to support.

That doesn't necessarily mean every requirement should be removed.

It means the system needs to account for the users who have to operate it.

In software, we call this accessibility.

We don't expect every user to interact with a system in exactly the same way. We provide alternatives, reduce unnecessary friction, and design interfaces around the capabilities and needs of the people using them.

Human services should be treated with the same principle.

If completing the process requires overcoming the same kind of barrier that the service is supposed to help with, then the architecture deserves examination.

You shouldn't need the outcome in order to have the capacity to navigate the process that produces the outcome.

Failure should be visible

A well-designed system makes failure observable.

If a dependency is unavailable, the system should say so.

If an input is invalid, it should explain why.

If a request cannot be fulfilled, it should fail clearly.

And if the system itself cannot provide the required service, that limitation should be visible as early as possible.

The alternative is a system where failure is hidden behind continued activity.

The user keeps attending appointments.

More forms are submitted.

More emails are exchanged.

The case remains open.

The process continues.

Eventually, months later, the system reaches the same conclusion it could potentially have reached at the beginning.

From the organization's perspective, there was plenty of activity.

From the user's perspective, there was no progress.

So, when is a system not fit for purpose?

I don't think the answer is simply "when the system isn't perfect."

No serious system is perfect.

And I don't think saying no is inherently a failure.

Sometimes the correct system behavior is:

We cannot help you with this.

The important part is what happens next.

A system becomes questionable when it cannot identify its own boundaries, cannot communicate them clearly, and can continue consuming a person's time and resources without producing a meaningful outcome.

In software, we'd investigate that.

We'd look at the architecture.

We'd examine the state transitions.

We'd inspect the failure modes.

We'd check the metrics.

We'd look at the monitoring.

And we'd ask whether the system is actually optimizing for the thing that matters.

Public services deserve the same level of scrutiny.

Because the ultimate test isn't whether the process ran.

It's whether the system achieved its purpose.

A system can be operational, compliant, fully staffed, and processing thousands of cases while still not be fit for purpose.

Sometimes the most important failure isn't that the system crashed.

It's that the system kept running exactly as designed while the person it was supposed to help went nowhere.

Learning to Build Without a Ladder

Doby Baxter — Sun, 06 Sep 2026 00:52:55 +0000

I never got the junior job.

Not for lack of trying. I spent well over a year applying, building, learning, and applying again. For long stretches I could not get past the first filter, let alone into a room with an engineer. And I am not unusual. I am one of a very large group of people who entered software and security over the past five years or so and found that the door marked "start here" had quietly been taken off its hinges.

We are often described as self-taught, as if that were a personality trait or a lifestyle choice. For most of us it was not a choice. It was the only path left standing once the others were dismantled.

The ladder that used to exist

The traditional way you became a good engineer looked roughly like this:

learn, get a junior role, make mistakes under supervision, become competent, become mid-level, eventually become senior.

Every rung mattered, and the early ones mattered most, because that is where you were allowed to be wrong cheaply. Someone experienced sat near you and said "not there, we tried that two years ago, here is what happened." That sentence is worth more than any tutorial ever written.

What replaced it, for a lot of people, is this:

learn, apply, rejected, apply, rejected, apply, rejected, somehow already be experienced.

And then the same industry turns around and says it needs experienced developers. Where, exactly, are they supposed to come from? You cannot eliminate the junior opportunities and then complain that there is a shortage of seniors. Those seniors had to be made somewhere. The industry has been eating its seed corn and acting surprised at the harvest.

The uncomfortable part is that no single company is behaving irrationally. Each one reasonably asks why it should train someone for two years when it can hire a person who already knows the system, especially when hundreds of thousands of experienced developers have been laid off and are already on the market. Individually rational, collectively ruinous. The full cost does not show up on a quarterly report. It shows up years later, when the people who could have become the next generation of seniors were never given the chance to start.

What I did instead

If the apprenticeship was gone, I had to build my own. So I went where the door was still open: open source.

Open source has always been one of the few places where you can say "I do not have a job yet, but I want to become good," and actually be taken up on it. You find a project. You read the code until it stops being frightening. You fix something small. You submit it. Someone reviews it and tells you, plainly, what is wrong. You improve. You come back for something harder. That loop is an apprenticeship. It is the apprenticeship, just without the salary or the org chart.

Here is what that looked like for me, concretely.

I contributed a fix to GitLab core that was reviewed, merged by a maintainer, and shipped to production in a release. Small in scope, but it was real code in a system that millions of people depend on, held to the standard of people who maintain that system for a living.

I built the Pyxel Config Lab, a tool for building and validating configuration for the European Space Agency's Pyxel detector-simulation framework. It was credited by ESA in an official release announcement. I did not have a job at ESA. I had a problem I understood and the willingness to solve it in the open.

I started contributing to OpenSPP, an open-source social protection platform, because reliability and validation are what I care about most and because that work touches systems that actually matter to people's lives. My first contribution there was, characteristically, about validation: making sure a record could not accept an impossible value and quietly produce nonsense downstream.

I began working into OpenTelemetry, because the instinct that made me care about clean configuration and honest error messages is the same instinct that makes me care about whether a distributed system, or an AI system, can tell you the truth about what it is doing.

Alongside that I built accessibility tooling as browser extensions, and I write. A lot. The writing is not a side project; it is how I think, and it is how a person with no institutional letterhead makes their work legible to strangers.

None of this was assigned to me. Nobody onboarded me. I assembled the apprenticeship out of whatever was reachable, and the reachable thing was the open commons.

What you can learn from home

Here is the part I want to be honest about, because I think the "just teach yourself" crowd gets it badly wrong in the other direction.

Information is everywhere now. You can learn languages, databases, distributed systems, compilers, networking, and architecture from your bedroom. What you cannot fully learn there is what it feels like to be responsible for a system that has been running for six years, with dozens of developers, three generations of architectural decisions layered on top of each other, real customers depending on it, and someone telling you not to touch a particular corner because the last person who did caused an outage.

That knowledge is situational. It is institutional memory. It is the difference between knowing what a database index is and having migrated a live database with users on it at an hour you would rather forget. No course, no book, and no chat assistant can hand you that. You have to live it, inside a real organization, with real consequences. Which is precisely the thing the closed door denies us.

So I am not claiming the self-taught path is equivalent. I am claiming it is what was left, that many of us became genuinely capable through it, and that the fact we survived the missing ladder does not prove the ladder was unnecessary. It proves how resourceful people get when a system fails them.

AI is an accelerant, and the direction is a choice

I work AI-assisted. I do not type every line unaided from a blank file, and I am not embarrassed by that. I remain accountable for every design decision and can explain any of them. For someone learning without a senior at the next desk, an AI collaborator is genuinely powerful: it will explain an unfamiliar architecture, propose alternatives, surface edge cases, write tests that expose your own assumptions, and shorten the hours you would otherwise spend stuck. That is real mentorship-shaped help, available to people the labor market gave nothing to.

But the tool is neutral, and the institutions using it are not. Picture two companies with the identical AI.

Company A says: apprentices use this, seniors review their work and teach them why things are wrong. AI becomes an accelerator for apprenticeship. You can fast-track a motivated beginner into a productive engineer faster and more cheaply than before, and they want to be there, so the company wins too.

Company B says: the AI writes most of the code, so we do not need juniors, give us a handful of seniors. AI becomes an accelerator for headcount reduction, and the learning layer is deleted.

Same technology, opposite outcomes. Company B looks better this quarter. Company B is also quietly ensuring there are no seniors to hire in five years.

There is a bitter twist here for people like me. The same industry that shut the door increasingly rejects the way self-taught engineers actually work. I have been ruled out of roles for using AI in an assessment, judged by a process that bans the exact tool the situation forced me to master, while the label "vibe coder" gets thrown at anyone who works this way. You cannot remove the apprenticeship, hand people a tool to survive without it, and then penalize them for having learned to use it.

The flood is fine. Judgment is the skill.

People worry about the flood: too many apps, too many libraries, too many books, too much of everything now that the barrier to creation has dropped. I do not share the worry. Lowering the barrier to making things is good. If someone who could never program before builds a small game for their friends, wonderful. Not every act of creation has to be economically optimized, any more than every painting has to sell.

The real line is not between AI-generated and hand-written, and it is not between accessible and gatekept. It is between creation connected to understanding and creation disconnected from it, in the places where correctness actually matters. Accessibility is wonderful. Learning is necessary. Blind delegation is dangerous. "Ban the tool" answers none of that. The skill that matters now is judgment: can the person understand, test, and improve what the tool produced. That is the thing we should be teaching, and it is exactly the thing you build by doing real work under review, which is exactly the thing being taken away.

Engineering is not a business subplot

Business constraints are real. Cost, deadlines, regulation, and risk shape engineering, and an engineer who pretends otherwise builds beautiful things nobody can afford. But there is a healthy boundary, and it has been crossed repeatedly. Business should set the constraints and the goals. Engineering should decide how those goals can be met safely and realistically. When leadership starts dictating technical reality instead, that this will take two weeks, that tests are optional, that the generated code is fine as-is, that another senior is an unnecessary expense, that the mentorship layer can go, you get exactly the collapse we are living through. A society can outsource the production of goods. It cannot indefinitely outsource the production of expertise.

The generation that learned to build without a ladder

That is not a happy ending, and I will not pretend it is one. The ladder mattered. Removing it hurt real people and is storing up a shortage the industry will feel for years. But it is also true that the commons is still open, that the work is still real, and that a reviewer on an open-source project will still tell you the truth about your code, which is more than most job applications ever will.

I am building my own path now as an independent engineer, and I try to pass the ladder back down where I can, including teaching basic digital skills to people who have even less access than I did. Not because the system worked, but because it did not, and someone has to keep the door propped open.

The question I would put to the industry is simple, and it is not "will AI replace programmers." It is this: who is going to produce tomorrow's experienced engineers, if today's companies keep removing every opportunity to become one?

Enforcing Data Invariants in Odoo: Model Constraints, Migrations, and Tests That Actually Cover the Fix

Doby Baxter — Sat, 05 Sep 2026 14:30:10 +0000

This post collects a few things I ran into while contributing fixes to OpenSPP, a social-protection platform built on Odoo. They are all variations on one question: when a value has to satisfy a rule, where does that rule need to live so it holds on every path the value can arrive through.

Write paths in Odoo

A stored model field can be written through several paths:

The form UI.
ORM create() and write() from other code.
CSV and Excel import (load()).
Remote writes over XML-RPC, the API, and DCI.

These do not share the same hooks. Where a guard is placed determines which of these paths it covers.

The relevant decorators

Three decorators come up, and the difference between them is the whole point:

Decorator	Fires when	Covers import / RPC / ORM
`@api.onchange("field")`	In the form UI only, as the user edits	No
`@api.constrains("field")`	On every create/write touching the field	Yes
`@api.depends("field")`	When a computed field recomputes	Yes, for stored computes

onchange runs client-side in the form. constrains runs server-side on the stored field. If a rule needs to hold everywhere, it has to be a constrains (or a database constraint), not an onchange.

Example: a future birthdate

res.partner.birthdate had a single guard, an onchange that reset the field if the date was in the future. That guard only runs in the form, so ORM writes, imports, and API writes all stored future dates. The non-stored age compute then rendered those as negative values.

The fix was to add a stored-field @api.constrains("birthdate"). Because the field is stored and writeable, the constraint fires on create, write, and import, so the rule no longer depends on which path the value came through. The onchange can stay as immediate form feedback, but it is no longer the only check.

Invalid state propagates to other fields

A second case shows why the location of the check matters beyond the field itself. A plan model had an is_current flag meaning "the current plan for this case." Completing a plan set its state to completed but never cleared is_current.

Downstream, several things read that pair:

A derived current_plan_id (based on is_current) kept pointing at the completed plan.
A "has an active plan" check read false.
A one-current-plan-per-case constraint then refused to let a new plan be marked current, because the completed plan still held the flag.

The incoherent value was written in one place and surfaced as a failure elsewhere: a user could not mark a new plan as current. The fix folded is_current = False into the same write() that sets the completed state, so every path that completes a plan also releases the flag.

A code fix only covers future writes

A write-time constraint or a corrected write() does not touch rows that are already in the bad state. Existing deployments still held completed plans flagged as current, and those cases stayed blocked after the fix shipped.

That needs a separate data migration. When the manifest version increases, Odoo runs matching migration scripts:

module/
  migrations/
    19.0.2.0.1/
      post-migration.py

def migrate(cr, version):
    if not version:
        return
    cr.execute(
        "UPDATE spp_case_intervention_plan "
        "SET is_current = false "
        "WHERE state = 'completed' AND is_current = true"
    )

Notes on the migration:

The signature is always def migrate(cr, version):, and cr is the raw cursor.
Keep the SQL literal. No f-strings, .format(), or composed identifiers, or the security linters (Semgrep, pylint-odoo) flag it.
The if not version: guard skips fresh installs, where there is nothing to repair.
Repairing existing rows and preventing new bad rows are two separate jobs. Both are needed.

Fixture and demo data can re-seed the bad state

The same module's demo data generator was creating the bad state directly (writing state = "completed" without going through the completion path, and passing is_current: True alongside a completed state). A migration does not help here, because a fresh install recreates the rows after the migration would have run.

The fix was to route both generator sites through the real completion action, so generated demo data satisfies the same invariant as production data. Worth checking the code that seeds data, not only the code that validates it.

Tests that pass without the fix

A test can go green for a reason unrelated to the code under test. Odoo applies field defaults before running validation, and _validate_fields fires constraints on the defaulted fields too.

In this model, registration_date defaults to today, and a pre-existing constraint requires registration date to be after birthdate. So creating a record with a future birthdate raised a ValidationError from that older constraint, before the new birthdate constraint ran. The test asserted only the exception type, so it passed with the new constraint removed.

Two ways to make such a test actually cover the fix:

Assert on the message, for example assertRaisesRegex(ValidationError, "Date of birth cannot be in the future"), so you know which constraint fired.
Or pass explicit values for the defaulted fields so only the constraint under test can raise.

A useful check either way: revert the fix locally and confirm the test fails. If it stays green, it is not covering the change.

It also helps to assert the reported symptom rather than the internal flag. If the bug was "a completed plan blocks marking a new one current," the test should complete the plan and then create a second plan and assert it does not raise, which is the behavior a revert has to break.

Date comparisons and timezone

fields.Date.today() returns the server date, which is effectively UTC. fields.Date.context_today(record) returns the date in the acting user's timezone.

For a "not in the future" check, the server date can reject a valid date for users ahead of UTC. A user in Pacific/Auckland at 10:00 local is already on the next calendar day while the server is still on the previous one, so a same-day birthdate compares as "in the future" against the server date. Using context_today(record) compares against the user's date instead.

Summary

For a rule that must always hold:

Enforce it with @api.constrains or a database constraint, on the stored field, not with @api.onchange.
Add a migration to repair rows already in the invalid state.
Check fixture and demo data so they do not re-seed the invalid state.
Write the test to fail without the fix, and assert the reported symptom.
Use context_today for date comparisons that depend on the user's calendar day.

If GOV.UK Knows the Application Is Invalid, Why Did It Take My Payment?

Doby Baxter — Fri, 21 Aug 2026 13:22:32 +0000

Maybe this is another one of those posts where I end up roasting the government. But stick with me, because this isn't really a politics post. It's a post about state machines, validation layers, and what happens when you put an irreversible action at the end of a workflow that never once checked whether the user should have been allowed to reach it.

The government part is just the catalyst. The engineering part is the point.

What actually happened

I applied for a British passport on GOV.UK.

Early in the form, it asks something like:

Do you have a certificate of naturalisation or a certificate of registration?

There are a lot of yes/no questions on that journey, and this one behaves exactly like the rest of them. I answered No, and the form did what it always does: it let me continue.

So I continued. I answered the remaining questions. I entered my personal details. I entered my bank details. The system accepted my payment of £107.

Only afterward did I learn that the certificate wasn't supporting evidence I could send in later. It was a prerequisite — something I needed before the application was valid at all.

I asked whether the application could be paused while I resolved the prerequisite. No. I was told I had roughly three months, and that the payment is non-refundable. If I can't complete everything in that window, the money is simply gone.

Here's the part that matters as an engineer: the system had my answer from the very beginning. I told it "No" on question three. It took my money on question twenty-something. Nothing in between used the information it already had.

The problem isn't the question

Asking "Do you have certificate X? Yes/No" is completely fine. That's not the bug.

The bug is what the system does after the answer.

A binary question that gates eligibility should not behave identically to a binary question that merely records a preference. If "No" makes the entire application invalid, then "No" is not informational — it's a hard gate. But the interface gave me no signal that I'd just walked through a gate. It looked like every other question, so I reasonably read it as every other question.

That's a semantic failure dressed up as a UI detail.

Input validation was never the issue

This is where my actual specialization kicks in — I spend most of my working life on configuration, error messaging, validation, and deterministic workflows — so let me be precise about which validation failed, because "the form should validate better" is too vague to be useful.

There are several distinct layers, and they are not interchangeable:

Syntactic validation — is this field shaped correctly? (Is the date a date?)
Semantic validation — does this value make sense in context? (Is the date in the past?)
Business-rule validation — is this allowed by the rules of the domain? (Are you old enough to apply?)
Workflow validation — given everything you've told me so far, are you eligible to be at this step at all?
Submission validation — is the application in a state where it can legally, financially, and administratively be finalized?

My passport form almost certainly nailed the first three. Every field was well-formed. Nothing was syntactically wrong.

It failed at workflow validation and submission validation — the two layers that actually protect the user from spending money on something that cannot succeed.

Don't let an invalid state propagate

Here's the conceptual shape of what should have happened:

START
  → eligibility questions
  → prerequisite missing
  → [ STOP: application cannot proceed ]

Here's what actually happened:

START
  → answer indicates prerequisite missing
  → continue
  → continue
  → continue
  → payment accepted
  → application submitted
  → "you cannot actually complete this process"

An invalid state that is detectable at step 3 should never be allowed to survive until step 20. In any other production system, we'd call that error propagation — a known-bad condition carried silently downstream until it detonates at the most expensive possible point.

The fix doesn't require anything clever. It doesn't need machine learning or a smarter form. It needs one correctly defined state transition:

if prerequisite_required and not prerequisite_present:
    block_submission()
    explain_why()
    explain_how_to_resolve()
    preserve_progress()

Never gate an irreversible action on an optimistic assumption

Payment is the textbook example of an irreversible action. Money leaves your account. The system tells you it won't come back.

So the condition for accepting payment cannot be:

payment_allowed = user_reached_payment_page

It has to be:

payment_allowed = application_state == READY_FOR_SUBMISSION

Reaching the payment screen is a fact about navigation. Being ready to submit is a fact about eligibility. The system conflated the two, and the user paid for the difference.

Any time an action is irreversible — payment, submission, deletion, dispatch — the system should treat it as a boundary that requires the whole prior state to be provably valid, not merely reachable.

The real world is asynchronous, and the workflow needs to model that

This is the part administrative software chronically gets wrong.

Completing a prerequisite in real life is rarely something the user can just do. It might depend on:

an appointment with a third party,
a document held by someone else,
a certificate that must be requested, issued, posted, scanned, and certified,
a process with its own multi-week turnaround.

None of that happens synchronously just because the applicant wants it to.

So a good workflow has to distinguish between two states that look identical on the surface but are completely different:

"The user hasn't done this yet." (Their responsibility. Nudge them.)
"The user cannot currently do this because another process hasn't completed." (Not their responsibility. Wait, don't penalize.)

A three-month non-refundable countdown that assumes every dependency is under the user's control is modeling the wrong reality. It's charging people for the latency of institutions they don't control.

Accessibility isn't only screen readers

Here's a distinction I wish more teams internalized: accessibility is not just a property of the UI. It's a property of the workflow.

A service can have perfect semantic HTML, keyboard navigation, ARIA labels, screen-reader support, and AAA contrast — and still be functionally inaccessible, because the underlying process is confusing, unforgiving, and impossible to recover from.

A workflow that quietly assumes users can instantly obtain every prerequisite, infer every unstated dependency, remember every required document, and correctly reinterpret an ambiguous yes/no question is already fragile for everyone. For anyone dealing with cognitive load, memory difficulty, limited resources, or dependence on third parties, that fragility stops being an inconvenience and becomes a wall.

You can pass every automated accessibility audit and still ship something that's impossible for a real person to complete safely. GOV.UK's design system is genuinely excellent at the component level. This failure lives one layer up, in the workflow, where the audits don't look.

What a fail-safe version would do

None of this is exotic. Concretely:

Dependency-aware questions — a gating answer visibly changes the journey instead of blending in.
Hard eligibility gates — invalid state stops at detection, not at checkout.
Explicit prerequisite explanations — say why you're blocked and how to fix it.
Persistent, resumable progress — don't force people to gamble against a clock they don't control.
Pre-payment validation — establish eligibility before taking money, always.
Asynchronous dependency states — model "waiting on someone else" as a first-class status.
Human escalation paths — a real route to intervene when the machine gets it wrong.
Rollback where the system permitted an invalid transaction — if the service let money through in a state it should have blocked, the refund is the system's responsibility, not the user's problem.

The uncomfortable question

So here's where I'll leave it.

If a system lets a user make an error, already holds enough information to recognize the consequences of that error, allows the error to propagate through the entire workflow, accepts payment, and then refuses to let the user recover — where exactly did the error occur?

It did not occur the moment the user clicked the wrong button.

It occurred when the system was designed without a safe state in which that mistake could be recovered from.

That's the bug. Everything else is just where it happened to surface.

A Floor Beneath Every Person: Design Choices in the First Social Resource Floor Blueprint

Doby Baxter — Fri, 14 Aug 2026 00:12:29 +0000

TL;DR — I've been building the Social Resource Floor: an open blueprint for coordinating one person's access to basic survival resources — food, housing, energy, healthcare, and more — across many independent providers, so that reaching those resources is grounded in being human rather than in financial access. The first blueprint version is now complete: language-neutral schemas, prose specifications, a reference implementation, and a first adapter. This post is about the engineering choices behind it, and the reasons for each — how it stays a contract rather than a product, how it keeps personal data out of the coordination layer, why it binds to existing standards instead of inventing new ones, and how I check that the contracts are implementation-independent rather than just claiming they are.

The problem the Floor is trying to help with

Today, for most people, survival routes through financial access. To reach food, housing, energy, or healthcare you generally need money, and to hold or move money you need banking, employment, or purchasing power. Financial access has become the gate standing in front of the resources a person needs to stay alive.

The goal of the Social Resource Floor is narrow and specific: to help make it so that financial status is not the condition that determines whether a person can reach the basic resources required to survive. It does not try to abolish money, banks, or markets — money stays a first-class resource and delivery method. It aims at one thing: a floor beneath which no person should fall, defined locally, reachable regardless of financial circumstances.

That's the mission. Everything technical below exists to make that mission buildable by the institutions — governments, municipalities, NGOs, cooperatives, community providers — that would actually run it, without asking any of them to give up their own systems or hand over their data.

Where the Floor sits

The delivery systems for social protection already exist and are strong: OpenSPP for orchestration, OpenG2P for delivery rails, OpenCRVS for civil registration. The Floor is not a replacement for any of them. It's a coordination layer above them, whose job is to let independent systems coordinate one person's whole survival floor — across many providers and many resource types at once — through shared, open interfaces, while each provider keeps its own mission and its own data.

The whole design follows from that position. A layer that sits above independently-operated systems and coordinates without owning them has a specific set of constraints, and each of the choices below is an answer to one of them.

Choice 1: Contracts first, implementation second

The source of truth is a set of language-neutral JSON Schemas. Prose specifications describe their intent, a conformance suite validates against them, and a reference implementation demonstrates them — but nothing in the repository is permitted to depend on that reference implementation. Every dependency arrow points inward, toward the schemas:

schemas/           source of truth — depends on nothing
  ^   ^   ^
  |   |   |
specification/   conformance/   reference/ + adapters/
  (prose)        (validates)    (demonstrate, non-authoritative)

The guiding rule is interface over implementation: the blueprint says what a component must be able to do, not which product must do it. The reason is adoption. If the reference implementation were the source of truth, then "conformant" would quietly come to mean "matches my code," and every participant would inherit my choices of language, database, and framework. Keeping the schemas authoritative and the reference deliberately non-authoritative means an independent party can build to the same contracts in a different stack and still interoperate. The concrete guardrail I hold myself to: if an adapter ever imports from the reference implementation, the dependency has inverted, and the project has started shipping a product instead of a contract.

Choice 2: A small data spine

The core data model is three nouns:

A provider publishes resources.
A resource is something a provider can supply — food, energy, housing, or money, which is one resource type among several rather than the organizing principle.
An entitlement allocates a resource to a subject and names the provider responsible for fulfilling it.

Around that spine sit six more schema areas — decision, provenance, event, consent, capability, and a public transparency report — for nine in total. Keeping the spine this small is deliberate: the fewer required concepts a participant has to adopt to join, the lower the barrier for a small community provider to become a full participant. Everything else is additive.

Choice 3: Keep personal data out of the coordination layer

A layer coordinating across many providers is a tempting place for personal data to accumulate, and that would be exactly the wrong outcome for a system meant to serve vulnerable people. So the design is data-minimizing by construction, not by policy.

The mechanism is a single field, subject_ref: an opaque, pseudonymous handle that means "the same subject, consistently, across the documents that need to correlate" — and nothing more. It carries no name, no cleartext ID number, no readable date of birth. Resolving it into actual facts about a person is done by systems that already do that — civil registries, national ID systems, credential issuers — entirely outside the Floor. The Floor never makes the resolve() call itself; it only ever holds the opaque result. Identity is treated as an interface with zero provider lock-in, the same way policy is.

This choice shows up throughout. The conformance suite resolves cross-references like a resource's provider_id or an entitlement's resource_id, but it deliberately never resolves subject_ref — there's nothing there to resolve, and that absence is the privacy property working as intended. Event envelopes, which are built to be broadcast between systems, carry the same discipline: subject_ref stays opaque and the payload carries coordination facts, never personal ones. Provenance records describe systems and operators, never subjects. A subject_ref is also not globally stable by default — a handle minted for one provider or jurisdiction need not be the same string used elsewhere for the same person, which limits how far any correlation can travel.

Choice 4: Bind to what exists; invent as little as possible

The most consistent decision across the specification is that before defining anything, I checked whether a mature standard or an existing system already solved it, and bound to that rather than inventing a parallel version. Each spec area records what was checked first. In practice that meant:

Policy / eligibility. The Floor does not define a rules language. It defines the interface a policy engine must sit behind — evaluate(policy_id, policy_version, jurisdiction, subject_context, resource?) -> DecisionRecord — and names existing engines as candidate bindings: OpenFisca for legislation-as-code, CEL (which OpenSPP already uses for eligibility), and OPA/Rego for authorization-style rules. The decision record's engine.binding field just names which one produced a given outcome. Any of them can be swapped without changing anything downstream.
Consent. Rather than a boolean flag on a subject's file, consent is modeled as records following the W3C Data Privacy Vocabulary and ISO/IEC TS 27560 — a status tied to a specific purpose, optionally narrowed to particular data categories or a particular provider/resource/decision. OpenSPP's own consent module is already DPV-aligned, so a participating deployment has something concrete to bind to.
Identity. Bound to DCI's typed-identifier lookups, W3C Verifiable Credentials with OIDC4VCI, and OIDC/eSignet/Keycloak as candidate resolvers — none mandated, all external to the Floor.
Authorization. The cross-provider "which system may do what" question binds to OAuth2 client-credential scopes, the machine-to-machine pattern OpenSPP's API and DCI already use. It's kept separate from OpenSPP's spp_user_roles RBAC, which solves a different problem — access control within one deployment, not between independent providers.
Interoperability and capability discovery. These follow the pattern G2P Connect already uses in production: named, versioned, independently-adoptable interface codes, where a participant implements only the codes relevant to its role and advertises which ones before an exchange begins.

The reason for this discipline is partly humility and partly durability: a coordination layer that reinvents consent, identity, and policy would be both arrogant about work others have done well and brittle against the systems it's meant to sit above. Binding to established interfaces means the Floor inherits their maturity and stays swappable as they evolve.

Choice 5: Append-only records, and reproducibility

Two smaller choices support auditability, which matters more than usual when the records decide what real people receive.

Records that change are never edited in place. A consent withdrawal is a new record with status: withdrawn and a supersedes pointer to the record it revokes, producing an append-only history a subject or auditor can walk. Provenance corrections work the same way — a correction supersedes rather than silently overwrites.

Decisions are built to be reproducible. policy_version is required on every decision record, because a decision that can't be re-evaluated against a fixed policy version can't be honestly audited later, whether for an appeal or a review. The same reasoning is why capability descriptors pin a schema_version: a version mismatch between two participants should be detectable before an exchange, not discovered as a validation failure after it.

Choice 6: Federation, and capability-based participation

The Floor is federated in the established sense — independent systems keeping control of their own operation and data, coordinating through shared open interfaces, in the pattern of email and the web where no one owns the protocol. It does not use or assume blockchain, ledgers, or tokens; "no single authority" refers to distributed governance, not to removing trusted parties, since providers, auditors, and operators remain trusted actors in accountable roles.

Participation is capability-based: a participant declares and implements only the subset of areas relevant to its role. A small community provider might only publish resources and consume entitlements aimed at it; a national ministry might publish resources, decisions, and entitlements while consuming consent records. Both are full, conformant participants. This is what makes the structure scale-independent — one provider on its own is a valid deployment, and it grows into a union of many providers without changing shape.

Checking that the contracts are implementation-independent

A blueprint that says "anyone can implement these interfaces" is making a claim, and I wanted that claim to be testable rather than taken on trust. That's what the conformance suite is for.

Fixtures are organized by intent: documents under valid/ must validate, and documents under invalid/ must be rejected, with each invalid fixture isolating a single violation so a failure points at one clear reason. The runner checks three layers, because the contracts are stricter than any single JSON Schema can express:

Structural — each document is validated against its area's schema (JSON Schema 2020-12).
Referential — cross-references between areas must resolve. A resource's provider_id, an entitlement's resource_id, provider_id, and decision_ref, and nested references like a consent record's scope.provider_ref all have to point at records that exist. subject_ref, as above, is intentionally excluded.
Semantic — invariants the schema standard can't state. The public transparency report enforces a small-cell floor: a breakdown bucket can't report a count below a declared threshold, because small aggregates re-identify people. JSON Schema 2020-12 can't require one number to be at least as large as a sibling field, so the suite enforces it in code.

The first blueprint version passes at 44 fixtures, 0 failures, and the full chain resolves end to end — a resource is published, a decision references it, an entitlement is drawn from it and pinned to the fulfilling provider — including a cross-provider case where a nationally-published resource is fulfilled by a different provider under the same shared floor.

To check that "implementable by anyone" holds rather than just "implementable by my checker," there are two independent conformance runners over the same schemas and fixtures: the reference is Python with the jsonschema library, and the second is Node with ajv — a different language and a different validation engine, chosen so that agreement can't come from shared machinery. A small script runs both and compares their per-fixture verdicts:

py="$(python3 conformance/runner/validate.py | verdicts)"
node="$(cd implementations/node && node conformance.js | verdicts)"

if [ "$py" == "$node" ]; then
  echo "cross-check: OK — both implementations agree on all $count fixtures"
else
  diff <(echo "$py") <(echo "$node")
  exit 1
fi

Right now both agree on all 44 fixtures, and I confirmed the mismatch path actually fails when fed a deliberate disagreement, so the check is doing real work. Alongside the runners, the reference implementation validates every document against the schemas on publication (it relies on the same contracts it demonstrates) and stores no subjects at all, and the first adapter maps OpenSPP's native concepts onto the provider and resource contracts with its output validated the same way.

What this version is, and what it isn't

To be precise about scope, since the mission is easy to overstate:

This version is a coherent, language-neutral contract for coordinating survival resources across independent providers; a conformance suite that makes "implementable" testable; and a demonstration, checked by two independent implementations, that the contracts rather than one codebase decide conformance.

It is not a running network, a governance body, or a funded pilot. Two runners I wrote agreeing is a strong signal that the schemas are implementation-independent, but it is not the same as two separate organizations interoperating in production. The next steps follow directly from that gap: an independent party running the same suite against their own output, and one small, honest pilot — a single municipality, a single resource type, one capability path end to end, consent-gated, with a transparency report — to show the small thing works before claiming the large one does.

That sequencing is intentional: build the shared understanding and the open contracts first, ship the smallest useful slice, and earn the standing to propose the larger structure to the institutions that would run it. Shared infrastructure of this kind rarely arrives because one person built all of it; it arrives because the groundwork already existed when a window opened.

Get involved

Repository: https://gitlab.com/dobybaxter127/social-resource-floor
Live site: https://social-resource-floor-5a9cc0.gitlab.io/
The best contribution at this stage is a hard question or a challenge to an assumption — governance, funding, and privacy are open by design.

References

Social Resource Floor — repository. https://gitlab.com/dobybaxter127/social-resource-floor
OpenSPP. https://openspp.org/en/
OpenG2P. https://www.openg2p.org/
OpenCRVS. https://www.opencrvs.org/
Digital Convergence Initiative. https://spdci.org/
G2P Connect. https://g2p-connect.github.io/
OpenFisca. https://openfisca.org/
W3C Data Privacy Vocabulary (DPV). https://w3c.github.io/dpv/
JSON Schema 2020-12. https://json-schema.org/

The Layer Above the Safety Net: Starting the Social Resource Floor

Doby Baxter — Mon, 10 Aug 2026 00:40:18 +0000

TL;DR — A day after I shipped my first contribution to OpenSPP, an open-source social protection platform, I couldn't stop looking up. Fixing a two-line bug meant tracing how one field moves through a system that decides what real people receive — and when I lifted my eyes from that field, I saw a gap that isn't a bug in any single project. There is no open, shared layer above the delivery systems that coordinates one person's whole access to survival resources, across many providers and many resource types. So I started one: a blueprint, not a product, called the Social Resource Floor. It is early, exploratory, and open. This is why it exists, what it is careful not to be, and where you can help.

It started one layer down

My first OpenSPP contribution was small on paper. A beneficiary's date of birth could be set to a date in the future, which made their computed age render as a negative number across the registry. The fix was a constraint. The education was everything around it: following every path that can write that field, and remembering that on a social protection platform a wrong age isn't cosmetic, because age feeds eligibility.

Doing that work put my hands on the machinery that decides whether a person is seen correctly by the program meant to help them. And it left me with a question that had nothing to do with birthdates. This system delivers benefits well within its mission — but what sits above it? Who coordinates when a single household needs food from one provider, energy help from another, and housing support from a third, across organizations that have never heard of each other?

The thing I couldn't unsee

Right now, for most people, survival routes through financial access. To reach food, housing, energy, or healthcare, you generally need money — and to hold or move money you need banking, employment, or purchasing power. Financial access has quietly become the gate standing in front of the resources a person needs to stay alive.

I want to be exact here, because this idea attracts a more dramatic version of itself that I don't hold. I am not saying money is bad. I am not proposing we abolish banks, markets, or the financial system. Those can keep existing and doing what they do.

The claim is narrower: financial access should not be the condition that determines whether a human can reach the basic resources required to survive. Money becomes one possible resource, and one possible delivery method — not the organizing principle of survival itself. That is a smaller thesis than "replace the economy," and a far more buildable one.

What the Social Resource Floor is — and what it refuses to be

The Social Resource Floor is an open blueprint for a coordination layer that sits above existing social-protection infrastructure. Its job is to let independent systems — government programs, municipalities, NGOs, cooperatives, community providers — coordinate access to survival resources through shared, open interfaces, while each keeps its own mission and its own data.

It is defined just as much by what it does not try to be:

Not a replacement for OpenSPP, OpenG2P, OpenCRVS, or any delivery platform. Those systems keep delivering; the Floor coordinates above them.
Not universal basic income, and not a bank, a government, or an NGO. It is a structure, not an institution that owns resources.
Not anti-money. Cash stays a first-class resource and delivery method.
Not, yet, working infrastructure. It is a blueprint — definitions, relationships, and interfaces — developed in the open.

Stating the negatives this loudly is the point. Ambition without boundaries is just noise; the boundaries are what make it possible to actually build something.

"Decentralized" was hiding four different questions

Early on I kept reaching for the word decentralized, and I kept confusing myself — until I realized the word was doing four jobs at once. Pulling them apart was most of the clarity:

Authority — who defines the rules? No single institution should govern every deployment; the floor is defined locally.
Operation — who runs it, and where? It should be locally deployable: anyone can run their own instance and stay interoperable with others.
Data — where does personal information live? It should be data-minimizing: data stays with the systems that already hold it, and only what a given interaction needs is exchanged.
Coordination — how do independent systems talk? Through open interfaces anyone can implement, with no central hub sitting in the path of every interaction.

Put together, that is not "decentralization" in the crypto sense. It is federation — the same pattern as email and the web, where independent parties implement open standards and no one owns the network. No blockchain, no tokens, no trustless consensus. Just open interfaces and independent systems that agree to speak them.

I expected a crowd. I found building blocks.

I assumed a hundred people were already building exactly this. What I found was better, and more humbling: a lot of serious, excellent work on the pieces, and an open question about the layer above them.

The delivery systems — OpenSPP, OpenG2P, OpenCRVS — are strong and explicitly interoperability-minded. There is real standards work underway too: the Digital Convergence Initiative is developing interoperability standards between identification and social-protection systems, with that same open stack in its demonstrations. None of this is siloed neglect waiting to be rescued, and I won't pretend it is.

What I didn't find was an open, shared abstraction for one person's whole survival floor, coordinated across many providers and many resource types at once. That is the specific question the Floor exists to ask. Framed honestly, it is less "nobody has done this" and more: these are building blocks — what does the layer above them look like, and can it be defined in the open?

Where it actually is right now

Early. Genuinely early. The current scope is deliberately conceptual, in four stages:

Define the concepts — a shared vocabulary for people, providers, resources, policies, and entitlements.
Map the relationships — how needs, eligibility, entitlements, and delivery connect across providers.
Find the interfaces — the minimum contract two independent systems need to participate.
Name the open questions — keeping governance, funding, privacy, and authority as explicit questions rather than buried assumptions.

How to get involved

This is exactly the stage where a few extra minds change the shape of a thing. Whether your background is social protection, engineering, policy, or lived experience of these systems, there is a way in:

Read the idea on the live site.
Read the README and specification drafts in the repository.
Open an issue with a question, a gap, or a hard challenge — governance, privacy, and funding are open by design.
Sharpen the models, or research adapters for OpenSPP, OpenG2P, and OpenCRVS.

Live website: https://social-resource-floor-5a9cc0.gitlab.io/
Repository: https://gitlab.com/dobybaxter127/social-resource-floor
Open an issue: https://gitlab.com/dobybaxter127/social-resource-floor/-/work_items

References

Social Resource Floor — live site. https://social-resource-floor-5a9cc0.gitlab.io/
Social Resource Floor — repository. https://gitlab.com/dobybaxter127/social-resource-floor
OpenSPP — Issue #362, the birthdate bug that started this. https://github.com/OpenSPP/OpenSPP2/issues/362
OpenSPP. https://openspp.org/en/
OpenG2P. https://www.openg2p.org/
OpenCRVS. https://www.opencrvs.org/
Digital Convergence Initiative — social protection interoperability standards. https://spdci.org/

The bug I started with was about one field being seen correctly. The Floor is the same concern at a different scale: whether a person can be seen — and reached — by the systems meant to help them, without money being the price of admission.

You Don't Validate in the Form — You Validate Where Every Write Path Converges

Doby Baxter — Sun, 09 Aug 2026 16:37:30 +0000

TL;DR — My first OpenSPP contribution fixes a small bug with a big shadow: a beneficiary's date of birth could be set to a date in the future, which made the computed age go negative and render as things like -3 across the registry. The web form already blocked it — but only the form. Every other way data enters the system walked straight past that guard. This post is about what I found when I followed each write path, why the one check that looked like it covered this was dead code, and the seven rules I now hold myself to about where validation actually belongs. The system is OpenSPP, an open-source social protection platform, which raises the stakes: on a platform like this, a wrong age isn't cosmetic — it feeds eligibility.

The myth I had to give up first

For a long time I believed that if the form validated an input, the input was validated. You type a bad value, the UI catches it, the user fixes it — done. That belief is comfortable because it's usually true from where you're sitting, which is in front of the form.

It stops being true the moment you remember that a form is one door into the building, not the building itself.

OpenSPP stores a beneficiary's date of birth on birthdate, and derives a non-stored computed age from it. In the web form, a small guard called _birthdate_onchange already reacts when someone types a future date: it clears the field and shows a friendly warning. If the form were the only door, this bug wouldn't exist. But a registry is written to by imports, scripts, and APIs far more than by people typing into a form — and none of them ring the doorbell.

So I stopped asking "does the form catch this?" and started asking the only question that matters for a system of record: does every path that can write this field catch it?

What I found when I followed every write path

I traced each way a birthdate can be set. The findings were consistent, and each one peels back the same assumption.

The onchange guard only runs in the form UI. This is not an OpenSPP quirk — it's how Odoo works. @api.onchange methods fire on form-view interaction and are never triggered by direct write calls. The friendly reset I was relying on is, by design, a UI convenience, not a data guarantee.

Every non-UI path skips it entirely. A future birthdate is accepted through direct ORM calls (create() / write()), through CSV and Excel import, and through the API surfaces (XML-RPC, API v2, DCI endpoints). Each of those is a door with no guard on it. In a social registry, those are the busy doors.

The check that looked like it covered this was dead code. There had been an @api.constrains("age") guard that appeared to validate exactly this. It never ran. age is a non-stored computed field, and a constraint on a field like that doesn't fire the way a constraint on a real, stored column does. It was removed in an earlier PR (#357) as a behavior-preserving cleanup — removing something that already did nothing. The lesson landed hard: a check that silently validates nothing is worse than no check, because it buys you false confidence.

Once stored, the bad value spreads. A future birthdate makes the computed age negative, and that -3 then renders in views, exports, and anywhere downstream logic reads the field. The invalid value doesn't stay where it entered; it leaks into every surface that trusts the registry.

And this is a social protection platform. A clean date of birth isn't a display detail here — age feeds eligibility and program logic. A negative age is, concretely, a person being mis-seen by the exact system meant to protect them.

Mapping what I found to rules I now hold myself to

Findings are only worth the tracing if they change what I do next time. Here's the translation, each rule tied to what I saw and to the fix I shipped.

Rule 1 — Validate where every write path converges, not at the door the user happens to use

From: the onchange guard runs only in the form; every other path skips it.

The form is one entrypoint; the model is where they all meet. So the guarantee has to live on the model. A stored-field constraint runs on create, write, import, and API alike, because all of them ultimately go through the ORM:

@api.constrains("birthdate")
def _check_birthdate_not_future(self):
    for record in self:
        if record.birthdate and record.birthdate > fields.Date.today():
            raise ValidationError(_("Date of birth cannot be in the future."))

Rule 2 — Constrain the input you store, not the value you derive

From: the dead @api.constrains("age") on a non-stored computed field.

The old guard tried to validate age, a derived value that isn't a real column — so it never fired. The fix constrains birthdate, the stored field the user actually writes. Validate the thing that gets persisted, and know which of your framework's hooks genuinely run versus which only look like they do.

Rule 3 — Keep the gentle guard and the hard backstop

From: the onchange's silent-reset UX is friendlier than a raised error.

The right move isn't to replace the form guard with the constraint — it's to keep both. The onchange stays as a soft, in-UI first line of defense that quietly corrects a typo before anyone hits save; the constraint sits behind it as the backstop that no import or API call can bypass. Defense in depth, not defense in replacement.

Rule 4 — When you refuse, speak the user's domain

From: the raw failure is a negative number and schema noise; the person needs a sentence.

A refusal has to say what's wrong in the user's world, not the parser's. The message is Date of birth cannot be in the future. — a plain, domain-level statement — not a schema violation about a stored column or a stray -3 surfacing three screens away. The registry's users think in beneficiaries and dates, so the error talks in beneficiaries and dates.

Rule 5 — A write-time constraint doesn't heal history

From: a constraint only validates on write.

Records that already hold a future birthdate stay invalid until they're next touched — and could then block otherwise-unrelated saves. So a constraint isn't the whole job for a deployed database; it wants a paired data-quality check or a migration note to find and fix the values that slipped in before the guard existed. Stopping new bad data and cleaning old bad data are two tasks, not one.

Rule 6 — Test every path you claim to cover, before the fix

From: the bug existed precisely because only one path was ever exercised.

I'm following the standard OpenSPP flow: write the failing test first, then the fix, and make the test cover create, write, and import — not just the form. A fix I only proved on the path that already worked would be repeating the original mistake in a nicer font.

Rule 7 — On a social platform, correctness is dignity, not polish

From: age feeds eligibility and program logic.

This is the spine the other six hang on. The reason to get a two-line constraint exactly right isn't tidiness — it's that a clean date of birth decides whether a real person is represented accurately to the program deciding what they receive. "Small bug" and "small stakes" are not the same thing.

The synthesis: validate where every path converges

Put it together and the shape is simple. A form guard answers "did this user, in this UI, enter something sane?" A model constraint answers "can this invalid value exist in the database at all, no matter who or what wrote it?" Those are different questions, and for a system of record only the second one keeps you safe.

The bug was never really about birthdates. It was about mistaking the door I could see for the whole building. The interesting part of the fix isn't the two-line constraint — it's understanding why the old guard never fired, where every write path enters the model, and how to protect existing data without breaking unrelated saves.

You don't validate in the form. You validate at the layer every path has to pass through — and you say what's wrong in words the person on the other end can actually use.

Takeaways

The form is a door, not the building. UI validation guards one entrypoint; a system of record is written to by imports, scripts, and APIs that never touch it.
Put the guarantee on the model. A stored-field constraint runs on create, write, import, and API alike, because they all converge on the ORM.
Constrain what you store, not what you derive. A check on a non-stored computed field can be dead code that silently validates nothing — the most expensive kind of check, because it feels safe.
Keep the gentle guard and the hard backstop. Defense in depth: a friendly onchange in front, an unbypassable constraint behind.
Speak the user's domain when you refuse. "Date of birth cannot be in the future," not a schema error or a stray negative number.
A write-time constraint doesn't heal history — pair it with a data-quality check or migration note for records that predate the guard.
On a social protection platform, correct data is a person's dignity. That's why a two-line fix is worth this much care.

References

OpenSPP — Issue #362, spp_registry accepts future birthdates via ORM / import / API. https://github.com/OpenSPP/OpenSPP2/issues/362
OpenSPP — PR #357, the behavior-preserving removal of the dead @api.constrains("age") guard. https://github.com/OpenSPP/OpenSPP2/pull/357
Odoo 17 — ORM API reference (@api.constrains, @api.onchange, computed fields). https://www.odoo.com/documentation/17.0/developer/reference/backend/orm.html
OpenSPP — the platform this contribution is for. https://openspp.org/en/

Paving the wrong Path: What a Broken Government System Taught Me About Feedback Loops

Doby Baxter — Mon, 27 Jul 2026 23:17:33 +0000

Last year I was made redundant from an NHS role. Around the same time, the team I worked in got a "new system." Everything that used to live on paper — or, honestly, in something that behaved like a shared Excel sheet — was going digital.

I was still employed during the first few weeks of rollout, so I watched it happen. And the thing I remember most clearly is that it didn't make anything better. The forms moved from paper to screen, and that was more or less the whole story. The same duplications, the same overlaps, the same unanswered questions. If anything, it felt more primitive than the retro spreadsheet it replaced.

I've been chewing on why ever since, and I think it's worth writing down, because the failure mode is one every engineer will recognize once it's named.

Paving the wrong path

There's an old phrase in software: if you digitize a bad process, you get a digital bad process.

The engineering equivalent is porting a legacy codebase line-for-line into a new language and calling it a rewrite. The syntax is new. The runtime is new. But you carried every piece of technical debt across the boundary intact, because you translated the implementation without ever revisiting the design.

Policy is the spec. Software is the implementation.

Here's the distinction I kept coming back to, and I think it's the crux of the whole thing. In a normal project, requirements drive implementation. The spec says what the system should do; the code is one interpretation of that spec. When they diverge, you have a defect.

In public systems, policy is the spec and software is the implementation. Every required field, every workflow step, every deadline, every automated decision is a concrete interpretation of a policy. This is where policy stops being an abstract document and becomes something a real person collides with at 4pm on a Friday.

So when a policy says "provide tailored support" but the software only ships a single hard-coded workflow, the implementation contradicts the spec. Nobody signed off on that contradiction. It just emerged from the gap between the two layers — the same way a bug emerges when the code does something the requirements never sanctioned.

A dependency with a bus factor of zero

The part that still bothers me most is how the system arrived. No manager knew who built it. Nobody could tell me where it came from, who maintained it, or how it fit the existing workflow. There was no training.

Frame that in engineering terms and it's absurd. This was:

an undocumented third-party dependency
deployed straight to production
with no README, no runbook, no changelog, no owner
and a bus factor of zero — not one person on site could answer a question about it

I only found out much later that it wasn't even a UK system; it was imported from the US. That's an unvetted external dependency shipped into a critical path, and the onboarding doc was the word "learn."

We'd never accept this in a repo. We'd block the PR. But institutions do it constantly, because the people deploying the system and the people using it are separated by enough layers that the missing context never registers as a problem — until it lands on the person doing the job.

Accessibility is an architecture problem, not a UI problem

When engineers hear "accessibility," we usually think keyboard navigation, screen readers, color contrast, text scaling. All important. But there's a broader meaning that matters just as much:

Can people understand what the system is actually doing?
Does it adapt to different circumstances, or force everyone down one path?
Is it clear who made a decision, and why?
Can someone challenge a decision without needing extraordinary time, money, or expertise?

Those aren't UI questions. They're architectural ones. And you can't fix an architectural problem at the presentation layer. If an API has terrible semantics, no amount of pretty documentation saves it. If the underlying service model assumes everyone follows the same workflow, a bigger button doesn't help the person the workflow was never designed for — it just makes the dead end easier to click.

The NHS system had this exact shape. The interface was new. The model underneath it still assumed one path for everyone, and that assumption is the thing that fails people.

The silent bug tracker

Now the part I care about most, because it's the one that generalizes to any organization.

A system can only improve if it's allowed to learn. And a learning loop needs three things:

Honest feedback — real signal about what's failing
A safe environment for producing that signal
A transparent process for deciding what happens with it

Break any one and the loop stops closing.

Engineers already understand this, because we understand observability. Feedback is telemetry. It's how a running system tells you where it hurts. Now imagine a project where every bug report is met with "why are you being so negative?" Give it a few months. The issue tracker goes quiet.

From the outside, that looks like stability. It isn't. People just stopped filing.

if report_is_honest:
    punish(reporter)

# a few weeks later
open_issues == 0
conclusion: "the system is stable"
reality: telemetry disabled, failures now silent

This is survivorship bias applied to an entire workplace. The dashboards are green because you turned off the sensors. And in a lot of public systems right now, honesty is punished — you get told off, told to get used to it, or told that's just how things are. So people stop believing they can change anything, and the signal dies. Not because everything works, but because reporting stopped feeling worth the cost.

There's a second, quieter failure hiding next to it: collecting feedback you never act on. We do this with metrics all the time — we track something because we can, not because we have a process to respond to it. A thousand survey responses that feed no decision are just vanity metrics. Gathering the data isn't the point. Closing the loop is.

Open source is the unlock — but policy has to evolve with it

I want to be clear: I think open source government infrastructure is genuinely one of the best things that could happen to public systems. For the first time, people who actually depend on a service — and the engineers who understand how it's built — can participate directly. They can read it, contribute, file issues, propose fixes, and have their needs acknowledged in a place everyone can see instead of scattered across emails, complaints, and tribunals. That's real power, and it's worth being excited about.

But — and this is the whole point — that power only fires if the policies evolve at the same time.

You can have public repos, open issue trackers, documented architecture, transparent decision logs — and still run a culture where nobody dares say "this isn't working," because the last person who did got punished. Open code, closed culture. The loop is still broken; it's just broken in public now. You've handed people a contribution channel and simultaneously made contributing unsafe.

So the two have to move together. Open sourcing the implementation while leaving the policy — and the culture around feedback — untouched just publishes the dysfunction. But open sourcing it alongside policies that actually welcome and act on honest input is where the potential becomes real: frontline staff and citizens stop being passive recipients and become part of how the system improves.

Because they're the ones hitting the failures every single day, their lived experiences hold the potential for your best telemetry.

The actual shift

Most conversations about digital government ask: how do we digitize existing processes? That question gets you paper-on-a-screen. It gets you the NHS rollout I watched.

The better question is: how do we build systems that continuously learn from the people who use them? That's not about replacing paper with software. It's about designing institutions with a working feedback loop built into the architecture — where the reasoning behind a system is documented, its assumptions are explicit, structural problems can be reported safely, and changes are visible instead of vanishing into private procurement and closed meetings.

Under that framing, publishing the code is almost the last step. The important part is making the decision-making itself observable.

And notice that none of it is really about technology. Python, Java, or paper — the values underneath are the same: transparency, accountability, shared responsibility, and a loop that's actually allowed to close. A system that only delivers a service is a system running blind. A system that learns from the people who build it and the people who depend on it is the one worth building.

Reconstructing cloud identity intrusions from O365 and CloudTrail logs

Doby Baxter — Mon, 20 Jul 2026 20:46:47 +0000

The uncomfortable truth of modern cloud IR is that there is often nothing to "detect" in the old sense. Attackers don't break in anymore; they log in. CrowdStrike's 2026 Global Threat Report put 82% of last year's detections in the malware-free column, with valid-account abuse behind roughly a third of cloud incidents. No dropper, no exploit chain — just a legitimate identity doing legitimate-looking things across Entra, M365, and a dozen SaaS apps.

That changes the hard part of the job. Detection tells you that something happened. It rarely tells you how: which account was really patient zero, which token got replayed, which consent grant quietly opened the door, and in what order it all unfolded. Responders reconstruct that story by hand, stitching a timeline out of fragmented, inconsistent audit logs, under time pressure, while the clock on breakout time keeps shrinking.

And manual reconstruction fails in a specific, predictable way: you anchor. You find the first narrative that fits the evidence, and you stop looking. The real path — usually the quieter one that doesn't announce itself — sits unexamined a branch away.

I built a tool to attack that failure mode head-on. It's called Nimbus Vestige, it's open source, and its defining feature is that it will tell you when it doesn't know.

Reconstruction, not detection

Nimbus Vestige (NV) is not a detector, a SIEM, a scanner, or an auditor. Those tools tell you that something happened and measure it. NV works backward — abductive reconstruction of the mechanism from patterns. Given normalized identity telemetry (sign-ins, consent grants, service-principal and role changes, mailbox access), it reconstructs the most likely path an intrusion took across accounts and services.

The one design decision everything else hangs off: it never hands you a single story. It hands you the ranked space of plausible paths. If your incident could have unfolded three ways, you see all three, ordered by how well the evidence supports each. That is the anchoring antidote — you can't tunnel on the first narrative when the tool is holding the alternatives up next to it. And the paths it didn't rank first but still rates plausible are a bonus deliverable: they're a hardening backlog, the other doors that were standing open.

The honesty problem

This is the part I care most about, because it's where reconstruction tools usually cheat. It is easy to emit a confident number. It is hard to emit a number that means something.

Every step NV reconstructs carries a confidence in [0, 1], built by evidence corroboration:

a per-technique base rate,
plus a bonus for each independent corroborating signal (source IP, device, successful outcome, temporal adjacency, broad-consent flags),
minus a penalty for signals that should be present and aren't (e.g. no source IP to corroborate an origin).

Path scores compose their links by geometric mean, not average — so one weak link honestly drags the whole path down instead of being smoothed over by strong neighbors.

Then there is a trust floor. Below it, a step is not shown as a guess — it is withheld, grayed out, explicitly unsupported. The floor is a slider: drag it up for strict, high-trust reconstruction, down for permissive, high-coverage. That is the honesty-versus-coverage tradeoff made physical, and it lives in the engine's output contract, not bolted onto the UI as an afterthought.

Do the base rates and bonuses actually mean anything? That's a calibration question, and I treat it as one. The weights start as defensible priors, and a harness refits them against labeled ground truth — events where you know which were on the attack path and which were benign — reporting Brier score, ECE, a reliability table, and a trust-floor sweep before and after. A refit is only adopted if it genuinely lowers calibration error rather than just shuffling numbers around. Calibrated confidence means one concrete thing here: NV's 0.7 should correspond to a step that really was on the attack path about 70% of the time.

The honest caveat, stated plainly because the whole point is honesty: the real calibration input is telemetry from an attack-simulation run (BadZure / MAAD-AF) in a live tenant. Until you point it at one, NV ships with a documented-shape proxy corpus, and every artifact fitted from that proxy is stamped as proxy in its provenance so it can never be mistaken for a live-tenant result.

One engine, more than one cloud

Every substrate-independence claim deserves suspicion until a second substrate exists. NV now has one: the same engine that reconstructs an Entra/M365 intrusion reconstructs an AWS one from CloudTrail.

The design that makes this cheap is that the engine core — the graph builder, the confidence model, the abductive layer, the trust floor — never knew anything about Microsoft in the first place. Only the vocabulary is provider-specific: which operation maps to which ATT&CK technique, and whether a step is an initial-access, pivot, or collection move. That vocabulary now lives in provider packs, and adding AWS meant adding a pack (IAM/STS/S3 → ATT&CK) plus a CloudTrail normalizer that emits the same event model. No engine change. The authorization gate generalized the same way: on AWS you scope-gate to an account instead of a tenant domain, and NV still refuses to run on any estate you haven't explicitly authorized.

Two details from the AWS eval are worth calling out because they're the honesty model doing its job on a new substrate, unmodified:

A deliberately-unknown IAM operation (PutRolePermissionsBoundary — no pack names it) survives ingestion, gets reconstructed abductively, and is flagged no known match at reduced confidence. The abductive layer doesn't care whose cloud it is.
An AWS ConsoleLogin lands below the trust floor that an Entra sign-in clears. That's not a bug — CloudTrail carries no device-posture signal, so an AWS login genuinely corroborates less, and NV refuses to pretend otherwise. A tool that fabricated a device signal to make the number look better would be exactly the kind of tool this project exists to not be.

The current AWS pack is deliberately narrow — IAM, STS, and S3 object reads, the identity-abuse core. It's a proof of generalization, not parity. Widening it is straightforward now that the seam exists.

Staying current without going blind

Every reconstruction tool meets the same two objections: your patterns will go stale, and you'll be useless on something novel. NV answers them on two independent fronts.

The known layer stays fresh through a validated update pipeline — MITRE ATT&CK (STIX), vetted CTI over TAXII 2.1, and the Sigma community ruleset, pulled on a per-feed cadence. Ingestion is treated as adversarial: every pattern, built-ins included, passes a trusted-source allowlist, a content checksum, schema validation, and an ATT&CK technique-id check before it enters the library, with an audit record for every admission and every rejection. An auto-update pipeline is a poisoning surface in disguise — a bad feed that injects false patterns manufactures false reconstructions — so the same skepticism the engine applies to evidence, it applies to its own patterns. (The feeds are scoped to the M365 beachhead they were built for; they refresh the Entra bindings and leave the AWS pack alone rather than over-mapping across substrates.)

The abductive layer covers what no feed has named yet. When an observed operation matches no known pattern, NV doesn't drop it — it reconstructs from first principles and flags it no known match at reduced confidence. The eval suite proves this twice over now: on a deliberately-unknown managed-identity token-theft step in the Entra corpus, and on the unknown IAM operation in the AWS one. Both caught, sequenced correctly, and honestly down-weighted. A signature engine goes blind on the zero-day; the abductive core degrades gracefully instead of going dark.

The ingest layer is built to serve that guarantee, not undercut it. The normalizers keep every identity-plane operation — including ones NV has never named — so a novel operation reaches the abductive core instead of being silently filtered out upstream. And they never fabricate a signal: if the source record carries no IP, no device, no consent marker, the field is simply absent, and the scorer reads absence as missing evidence rather than inventing corroboration.

Try it on public data

The demo needs neither your tenant nor anyone's cooperation. It runs on the public invictus-ir Office 365 unified audit log corpus, so you can see the whole pipeline end to end:

git clone https://gitlab.com/dobybaxter127/nimbus-vestige
cd nimbus-vestige
# open nv_gui.html — it renders the bundled demo reconstruction

When you point it at real data, that data never leaves your environment — NV is fully self-hosted. You export the M365 unified audit log (CSV with the standard AuditData column, or JSON/JSONL), normalize it, and run the engine scope-gated:

# 1. normalize raw O365/Entra audit logs into the event model
python3 nv_extract_identity_events.py auditrecords.csv nv_identity_events.jsonl

# 2. reconstruct — scope-gated to a tenant you are authorized to investigate
python3 run_nv.py nv_identity_events.jsonl reconstruction.json \
    --trust-floor 0.6 --authorize your-tenant.onmicrosoft.com

# 3. view — open nv_gui.html and click "load reconstruction.json"

The GUI is a pure view layer: load the file the engine just produced and the same screen shows your incident, your entities, your confidence bands — and every dot on the embedding canvas re-runs the reconstruction for that entity on click, so pivoting from "the incident" to "what did this account actually do" is one click, not a re-run.

The AWS path is the same three steps against CloudTrail:

python3 nv_extract_cloudtrail_events.py cloudtrail.json aws_events.jsonl
python3 run_nv.py aws_events.jsonl reconstruction.json --authorize aws:123456789012

What it isn't, and where it's early

Being straight about the boundaries, since I'd want the same from anyone else's tool:

It's not a detector or a SIEM. It comes in after something is flagged, to explain how the pieces fit. It will not alert you.
It's identity-first. Entra/M365 is the deep substrate; AWS is in as an eval-proven second one, but its pack is IAM/STS/S3 only so far. Wider AWS coverage and a third substrate (GCP, Okta) are the next steps — and network, endpoint, and OT remain roadmap.
Calibration is currently demonstrated on public / proxy data. Live-tenant weights are what you would actually deploy, and that needs a run in your estate.
It's built by one person and it's early. That is exactly why I'm posting this.
Live demo: https://nimbus-vestige-88b564.gitlab.io/
Repository: https://gitlab.com/dobybaxter127/nimbus-vestige

The core engine, both provider ingests, GUI, and offline eval harness are free and open. There's a one-time commercial license for teams that want the curated live feeds, calibration tooling, and support — but nothing about that gates the core, and nothing asks you to send your telemetry anywhere.

AI as a Laundering Machine: What Software Engineering Taught Me About Accountability

Doby Baxter — Sun, 19 Jul 2026 14:04:19 +0000

If an employer uses AI to decide whether I'm worth interviewing, we call it innovation. If I use the same technology to write more clearly or work around a disability, we call it cheating. The algorithms haven't changed — only who benefits.

The paradox

There's an asymmetry in how we talk about AI and work that I can't stop thinking about, because when an employer runs ten thousand applications through a ranking model, we call it efficiency. When an applicant uses that same technology to organize their thoughts or compensate for a cognitive barrier, the language suddenly turns to authenticity, fairness, and "unfair advantage."

The technology hasn't changed. The algorithms haven't changed. Only one variable has: who benefits from the automation.

As a software engineer, that asymmetry tells me our real disagreement isn't about AI at all. It's about who is allowed to use technology to amplify themselves, who has to justify it and who is answerable when automated systems shape someone else's life.

Bias isn't the interesting question

Most debates about AI in hiring ask whether the algorithms are biased. Fair enough, but bias has existed in hiring for as long as hiring has. AI didn't invent discrimination. The more profound question is what AI does to accountability. AI can become a laundering machine. I use the word deliberately. Money laundering doesn't create dirty money, it changes its appearance until the origin is impossible to trace. The reality stays the same; the provenance disappears.

A recruiter who rejects you leaves a chain of responsibility: someone read the application, someone reached a conclusion, someone exercised judgement. Now route that same rejection through applicant tracking systems, scoring models, ranking algorithms, and configurable thresholds. The outcome is identical, but the decision appears to have no author. "A recruiter rejected this" becomes "the model assigned a low suitability score." Judgement becomes an optimization objective. Discretion becomes probability. The rejection hasn't vanished; it's been dressed in the vocabulary of mathematics until it looks objective.

That's the laundering machine. It doesn't wash away prejudice. It washes away prejudice's apparent origin. The outcome remains real. Only responsibility becomes hard to see.

Every abstraction hides causality

Here's where software engineering has something to say. Our entire discipline is built on abstraction. We hide complexity behind interfaces, APIs, frameworks, and platforms so that each engineer can reason about one problem without drowning in all the others. Abstraction is a force multiplier — modern software couldn't exist without it.

But every layer that hides complexity also hides causality. We trade visibility for manageability, and most of the time that's a great deal. A web app calling an API doesn't need to know which processor ran the request. Responsibility is different. A software component can safely hide its internals because its behavior stays observable through its interface. Responsibility doesn't work that way. Put enough distance between a consequential decision and the humans who shaped it, and it stops being clear who exercised judgement, who accepted risk, and who should answer for the outcome.

We already know this instinct in engineering. We'd never run production without logs, traces, and deployment histories — because once causality goes opaque, reliability becomes impossible. Yet when automated systems shape human decisions, we routinely accept a level of opacity we'd never tolerate in our own software. Observability can't be bolted on after a system fails. Accountability follows the same rule. If a socio-technical system isn't designed to preserve attribution from the start, responsibility won't magically appear later.

Nobody decided, but the system did

Modern hiring isn't a recruiter at a desk. It's a socio-technical system: people, software, policies, incentives, and models whose collective behavior decides who gets access to work. Distributed systems engineers know how these behave. A latency bug in one microservice cascades through a queue, triggers retries, saturates a database, and surfaces as an outage somewhere completely different. Every component behaved to spec. The system as a whole did something no one designed.

Hiring works the same way. The feature engineer optimizes predictive performance. The data scientist evaluates accuracy. The product manager improves workflow. HR reduces cost. Procurement picks a proven vendor. Every participant can honestly describe their contribution without ever saying "I decided this person shouldn't get an interview."

Yet collectively, the system decided exactly that.

This is where distributed responsibility gets mistaken for absent responsibility. But there's an old rule in engineering: someone owns every service. When an outage hits, we don't shrug and say the system has no owners because many people built it. We investigate. We trace root causes. Complexity raises the effort of attribution — it never removes the need for it.

Human consequences deserve no lower standard.

Proxies inherit the past

You can't optimize for "find the best person" — the phrase is too vague to be an objective. So, exactly as we do everywhere in engineering, we optimize proxies: years of experience for competence, continuous employment for reliability, prestigious universities for ability.

Individually, none of these are unreasonable. The problem is that a proxy always discards information — and quietly inherits the assumptions baked into the history it was derived from.

If an organization historically overlooked people with employment gaps, unconventional backgrounds, or disabilities requiring flexible arrangements, those decisions become training data. The model doesn't understand the social circumstances behind the pattern. It just observes that certain applicants were historically preferred, and faithfully reproduces it because doing so improves its score.

From the model's view, these patterns are predictive. From a human view, they may just be historical exclusion.

And it's self-reinforcing: human judgement becomes training data, training data becomes a model, the model shapes new decisions, and those decisions become tomorrow's training data. "All models are wrong, but some are useful" — the danger isn't that the model simplifies reality. It's that we stop noticing what got discarded in the simplification.

"Human in the loop" can be theatre

The standard reassurance is that there's "always a human in the loop." From an engineering perspective, that tells us almost nothing about how the system behaves. Inserting a step into a workflow doesn't guarantee oversight. A deployment approval only matters if the reviewer has the information to spot risk, the authority to say no, and the time to actually look. When volume outpaces review capacity, approval becomes procedure, not judgement.

We even have a name for the failure mode: automation bias. Once software earns a reputation for being "usually right," reviewers start confirming its outputs instead of questioning them. The human quietly shifts from making decisions to rubber-stamping them. There's a useful distinction here from distributed systems: monitoring a process is not controlling it. A recruiter watching an algorithm's recommendations isn't necessarily exercising control over its influence. So "was a human involved?" is the wrong question. The right one is: where did human judgement actually enter, and did meaningful authority survive once the recommendations were generated?

Accessibility is not an unfair advantage

Now flip the paradox around.

When engineers optimize a system, we distinguish between removing overhead and increasing capability. Stripping out unnecessary overhead lets existing resources do more — it doesn't magically upgrade the hardware.

Accessibility works the same way. Adaptive technology doesn't elevate someone beyond their ability; it removes irrelevant barriers that were obscuring that ability in the first place. A programmer's competence isn't their typing speed. A researcher's insight doesn't depend on whether they dictate or type. Which brings us right back to the opening asymmetry. An employer using AI to process ten thousand applications is "innovating." A disabled applicant using AI to organize their thoughts before submitting is "gaining an unfair advantage." Same technology, same computational principles — one reduces organizational workload, the other reduces human barriers, and we embrace the first while treating the second with suspicion.

Accessibility has never been about lowering standards. It has always been about removing obstacles that stopped people from meeting those standards on equal terms.

Responsibility cannot be virtualized

Every mature engineering discipline eventually learns the same lesson: a system can't be trusted merely because it functions. It has to be observable, auditable, and accountable. The law is society's version of the same instinct — a mechanism for preserving provenance, so that decisions can be traced back to the people and duties behind them.

Employment decisions deserve no less. If an automated system rejects you, meaningful accountability means more than knowing software was involved. It means being able to reconstruct which data informed it, which policies shaped it, who set the thresholds, and who accepted responsibility for deploying it. Complexity doesn't reduce that need. It strengthens it.

Because here's the thing responsibility is not: it's not a computational resource. Software distributes computation across thousands of processors. Cloud platforms abstract away infrastructure. But responsibility can't be containerized, orchestrated, or load balanced until no identifiable owner remains. Someone still decided what to build, why, how it should operate, and what risks were acceptable. Those decisions stay irreducibly human.

Algorithms have no moral agency. A neural network cannot appear before a court, defend a hiring policy, or apologize to someone whose life it altered. The danger was never that computers start acting like people. It's that people start believing responsibility belongs to the system rather than to themselves.

So if one idea survives this whole essay, let it be this:

The true measure of an intelligent society is not how successfully it automates decision-making. It's how carefully it preserves human accountability after those decisions have been automated.

Responsibility cannot be optimized away. It cannot be abstracted away. It cannot be virtualized. Because the legitimacy of every system we build depends, in the end, not on the intelligence of its algorithms — but on the willingness of the people who build, deploy, and govern them to remain visibly, demonstrably responsible for what follows.

The Safety Net Is an Accessibility Feature: ADHD, AI, and How I Actually Build Software

Doby Baxter — Fri, 17 Jul 2026 20:07:29 +0000

A while back I wrote that you don't need perfect code — you need a safety net that talks back. People read it as an engineering post, which it was. But I left something out. I don't build those systems only because they're good engineering. I build them because they're how I stay functional as a developer with ADHD and autism. The clear error messages, the validation that runs before anything else does, the tests that fail loudly and specifically — those aren't just reliability features to me. They're external memory. They're the structure my own brain doesn't reliably hold on its own.

I think good engineering and accommodation are, quietly, the same thing. Here's what I mean.

What breaks me isn't difficulty. It's silence.

The hard part of a bug, for me, is almost never the logic. I can sit with a hard problem for a long time. What wrecks me is ambiguity: the silent failure, the config that loads fine and then falls over three steps later with no clue why, the "it just doesn't work" with nothing to grab onto.

That's not a character flaw and it's not laziness. It's what happens when a system asks me to hold its entire state in my head at once. My working memory doesn't do that well. So a system that fails quietly is asking me for something I can't reliably give and a system that fails loudly, early, and specifically is handing the context back to me instead of demanding I reconstruct it. Here is the difference a message makes:

This is the whole thing in miniature. Two versions of the same failure:

Error: invalid configuration

versus:

Error: 'timeout' must be a positive integer (got: "30s")
  in pipeline.yaml, line 14
  hint: use a number of seconds, e.g. timeout: 30

The first one asks me to become a detective. The second one just tells me what's true and where to go. For any developer that's a nicer experience. For a brain that struggles to hold a stack of context, it's the difference between staying in the work and losing the whole thread.

Once you notice that, you start seeing accessibility everywhere in ordinary engineering. Good defaults reduce decisions. Loud tests remove the "did I break something?" background anxiety. Reproducible environments mean I don't have to remember the fourteen steps that make the thing run. None of that is a special accommodation bolted on for disabled developers. It's just good design that happens to help most where people have the least slack to spare.

Where AI actually fits

AI landed in the middle of all this as, essentially, one more thing that talks back. And I want to be honest about it, because the hype and the doom are both exhausting. Where it genuinely helps:

Activation energy. The blank page is a real, physical barrier for me. Getting from nothing to something I can react to is the expensive part, and reacting is much cheaper than initiating. A rough first draft I can push against gets me moving in a way willpower never has.
A software that answers. Talking through a design out loud, even to a model that's sometimes wrong, helps me externalize the thought instead of looping on it internally.
Turning vague intent into a starting structure. "I want a validator that does roughly this" into a skeleton I can then correct. The correcting is where I'm actually good.

But also AI is confidently wrong in exactly the way that hurts me most. It papers over the silent-failure problem by inventing a plausible answer, which is the one thing my whole workflow is built to prevent. So I don't let it replace the safety net. I put the safety net around it. Types, tests, and validation gatekeep its output the same way they gatekeep mine. It's a drafting partner working inside a system that checks its work. That distinction matters more for me than for most, but I'd argue it's the healthy way for anyone to use these tools. The model isn't the source of truth. The system that verifies the model is.

The pattern underneath all of it

If disability taught me one thing as an engineer, it's this: don't rely on willpower or memory to hold a system together. Externalize it. Make the system carry the load, so that being tired, or distracted, or in a fibro flare, or just human, doesn't quietly break everything.

For a long time I thought of it as the compromised version of how a "real" developer works. I don't anymore. It's just good design. The system that fails loudly, explains itself, checks its own state, and doesn't depend on a perfect operator is more reliable for everyone who touches it. Building for the constraint made the thing better in general.

There's a well-worn version of this idea: curb cuts were fought for by wheelchair users and now everyone with a stroller, a suitcase, or a bad knee uses them without thinking. Software is full of curb cuts waiting to be built. A clear error message is one. A config that validates before it runs is one. An environment that sets itself up is one.

The part I don't usually put in my bio

I'll be plain about this, because pretending otherwise doesn't help anyone reading it who's in the same place. I've been rejected by essentially every traditional developer route I've tried. And the one role I did land, I didn't lose because I couldn't do the work, I lost it because the environment couldn't bend even slightly to how I work. No adjustments, an inflexible structure, and a slow, predictable slide into burnout that had nothing to do with the code I was writing. For a long time I read that as a verdict on me. It isn't. It's a mismatch between real ability and a role built to accept only one shape of person.

And here's the thing I most want to land: asking for adjustments is not asking for charity. Nobody calls a good error message a favor. Adjustments are simply the conditions under which I can do the work at all. Predictable hours, a manageable load, an environment that doesn't run me into the ground — those aren't perks that make a comfortable job nicer. They're the difference between me producing something real and me producing nothing. Remove them and you don't get a diminished version of the work. You get no work, from someone who was entirely capable of doing it.

That's the same principle as everything above. Externalize the structure. Don't build a system — or a job — that only functions if the operator runs on willpower and never has a bad day.

Building sideways

Here's where I've landed, and I don't say it with bitterness. We all function differently. Until the job market becomes more humane about that, until "can you do the work" stops being quietly replaced by "can you do the work in exactly this shape, these hours, this room", a lot of us are going to have to build sideways. Make our own way in. Contribute through the routes that don't have a broken filter standing at the door. That's what open source has been for me. It's how a merged contribution to a tool used across the industry, or a tool released alongside a European Space Agency project, exists at all not because a hiring funnel let me through, but because the work itself was open to anyone willing to do it. Non-traditional isn't lesser. For a lot of us it's the only door that was never locked.

And there's an irony I can't get past. So much of this field was built by minds that work the way mine does, the deep, narrow focus; the pull toward systems and patterns; the need to understand a thing completely before trusting it. The traits being screened out at the top of the funnel now are, in a lot of cases, the same traits the industry was built on. We didn't wander into tech by accident. In a real sense the culture of it was ours first. Being filtered back out of it stings precisely because of that. I don't have a tidy fix for an entire hiring market. But I know what I can do: keep building in the open, keep making the systems I touch talk back, and keep writing this down — so the next person who's heard "no" more times than they can count knows the problem was never them.

What I'd actually leave you with

You don't need perfect code, and you don't need a perfect brain either. What helps — what actually holds up on a hard day — is a system that talks back:

Make failures loud, early, and specific. Assume the person reading the error has zero context, because sometimes that person is you in six months, or you at 4pm with nothing left in the tank.
Externalize state instead of trusting memory. If the setup has fourteen steps, that's fourteen chances to fall off. Automate them.
Let AI reduce activation energy, but never let it be the safety net. Put the checks around it.
Design for your worst day, not your best one. The version of you with full focus and energy doesn't need the help. Build for the other version.

None of that is charity for the neurodivergent. It's how I ship reliable software, and the fact that it also makes the work survivable for a brain like mine isn't a coincidence. It's the point. The safety net was an accessibility feature the whole time. I just didn't have the words for it until I stopped being embarrassed about needing one.

I'd really like to hear from you on this one. If you build differently because of ADHD, autism, chronic illness, or anything else about how your particular brain and body work, what's the adjustment or the habit that actually makes the work possible for you? And if you've had to build sideways into this industry rather than through the front door, I'd love to know what that looked like. Drop it in the comments. I read all of them, and I suspect a lot of us are quietly solving the same problems alone.