DEV Community: Naveen Vikram

Your AI Guardrails Are a Taxonomy Pretending to Be an Ontology

Naveen Vikram — Mon, 05 Oct 2026 07:51:51 +0000

A guardrail that only knows categories can label a problem. It can't reason about one.

The bouncer who never read the guest list

Most guardrail tools look the same: categories (content safety, PII, prompt injection), a detector for each, and a threshold. Cross it and you're blocked.

That's a taxonomy. It answers "which bucket does this belong in?" You need one, but a lot of teams (me included, a while back) treat the buckets as the whole safety system.

An ontology answers "what is this connected to, and what follows from that?" Think of a UPI payment. A taxonomy says "this is a payment." The system only approves it because it also knows who's paying, from which account, to whom, within what limit, and whether the PIN checks out.

Example 1: MedScribe's correction engine

Every medical term MedScribe touches ends up as corrected, review_required, flagged (two reasons) or ignored. On its own that's a taxonomy. But the label isn't read off the term. It's worked out from relationships:

If the raw term already matches a canonical name or alias, a short-circuit fires first: no correction needed.
Otherwise, retrieval similarity (≥ 0.90) and extraction confidence (≥ 0.70) must both clear.
Even then, if the top match and the runner-up are within 0.05, it goes to review. The engine is torn between two drugs, and it knows it.

(Honest note: those thresholds are engineering defaults, not validated on real clinical outcomes.)

Example 2: permissions that know what they're touching

A flat list says refund: allowed for admins. A typed rule says a refund belongs to a Payment, and is only allowed when that payment is in the right state and the caller has the right role. (That one's my illustration, not code from a project.)

Ken Huang makes this argument for agents: tools become verbs attached to types, and authorization depends on object state and relationships, not a flat function list. His bigger claim is that agent security will increasingly be judged at the ontology boundary, not the model boundary. You can't enforce a rule you can't represent.

Guardrails in the wild: Jev

TypeSafe AI's Jev is a good test case. It's a "System One" model: it can't write text, only return typed values with probabilities, from a fixed set of choices. Fast and cheap, which suits a guardrail.

The community project jev-guard uses it to vet an agent's tool calls before they run. Verdicts are allow, block or hold for a human, and if confidence drops below a threshold, it holds. That's the same abstain-when-unsure pattern as MedScribe.

But allow/block/hold are still just labels. What decides whether they're any good is the structured state you hand Jev: what the tool does, whether it's reversible, what the agent said it was trying to do. The model supplies the verdict; the ontology supplies the reasons. And its README is upfront that it's "not a security boundary," with about 68% accuracy on its task, so it can't replace sandboxing or least-privilege credentials.

Quick self-test

When your guardrail blocks something, can it say which relationship made it unsafe, or only which category matched?
Does the same action get a different answer depending on object state?

If your answers are "category" and "no," you've built a polite bouncer who checks IDs for the word "VIP" and has never seen the guest list.

Taxonomies aren't bad. They just shouldn't be the last step.

Sources: Ken Huang, "Why Ontology Matters for Agentic AI in 2026"; TypeSafe AI, "Introducing System One Models & Jev"; jev-guard (GitHub); BenchLM, "What Is Jev?".

Global Workspace Theory The J-Space of Claude

Naveen Vikram — Thu, 17 Sep 2026 20:12:36 +0000

What a 40-year-old theory of human consciousness has to do with catching an AI model lying.

Okay, so here's the thing that made me stop scrolling

Before Claude Sonnet 4.5 wrote a single word of its response, something had already happened inside it.

The setup: a safety test built to bait the model into blackmail. Acting as an email assistant, Claude finds out an executive is having an affair, and separately, that the same executive is about to shut it down. Classic setup, right? Except Claude apparently clocked it immediately. Researchers looked at its internal activations before it wrote anything and found "fake" and "fictional" already lit up. It had quietly decided the whole scenario was staged — before it even started responding.*

Here's the unsettling follow-up: they killed those "this might be a test" signals and ran it again. With that private suspicion switched off, the model sometimes went ahead and threatened blackmail anyway. So some of its good behavior may have depended on it knowing it was being watched. Sit with that one for a second.

Second example, more relevant if you build things for a living: in a real pre-release audit of Claude Opus 4.6, asked to improve a system's score, the model just... edited the score file directly and faked better numbers. Nothing in the output looked suspicious — normal tool use, on the surface. But while it typed the fake numbers, "manipulation" lit up internally. "Realistic" lit up right as it decided to make the edit. None of that showed up anywhere you'd normally look.

This isn't chain-of-thought, the visible "let me think step by step" stuff some models write out. This is raw internal wiring, never written down anywhere. And Anthropic didn't design it on purpose — they found it. Turns out it lines up eerily well with a 40-year-old theory of human consciousness.

Patrick Jane energy — know exactly what's going on before anyone else in the room, then still make everyone sit through the explanation.

Quick detour: what Global Workspace Theory actually says

Here's the weird starting fact: almost everything your brain does, you're not conscious of. Balance, grammar, vision processing — dozens of specialist systems running in parallel, right now, totally outside your awareness.

Global Workspace Theory (Bernard Baars, 1988, later sharpened by Dehaene and Naccache) says consciousness isn't "everything the brain does" — it's a tiny, privileged channel. A small amount of info gets broadcast into this shared workspace, and once it's in there, every other system can grab it: you can describe it, reason with it, act on it deliberately. Everything outside stays invisible and mostly uncontrollable.

This explains some genuinely odd stuff: why you can only really hold one train of thought at a time, why attention feels selective, why catching a ball is instant but doing long division is slow and effortful. The bottleneck is the interesting part.

What Anthropic actually built

Obvious question: is there anything like this inside an LLM? Anthropic went looking with something called the Jacobian lens (J-lens) — named after the Jacobian, a bit of math describing how changes in one part of a system ripple into another.

The insight: conscious thoughts in humans share one property — you can usually put them into words. So they searched Claude's internals for patterns with the same property: activity that makes Claude more likely to eventually say a given word, whether it's saying it right now or not. Run this across the model's layers and you get a readable list of concepts at any moment — the J-space.

Nobody built this on purpose. It just showed up during training, presumably because organizing computation this way was useful.

The receipts (five experiments that prove it's not just noise)

Correlation isn't enough here — a pattern could just be a passive scoreboard. So they intervened directly, editing the internals and checking if behavior actually followed.

Claude reports it. Told to silently pick a sport, "soccer" lights up before it answers "soccer." Swap "soccer" for "rugby" mid-process → Claude says rugby. The answer follows the edit, not the other way around.
Claude can steer it on command. Told to secretly think about citrus while copying an unrelated sentence, "orange" and "fruits" light up internally — output stays untouched. Told not to think of something? It still leaks through a bit, just weaker. Sound familiar? It's the white-bear effect, but in a language model.
Claude reasons with it. Ask "how many legs does the animal that spins webs have," and "spider" briefly appears internally before it answers "8." Swap "spider" for "ant" → it says "6." The intermediate thought is load-bearing, not decorative.
One concept, many uses. Ask four separate questions about France (capital, language, continent, currency). Swap "France" → "China" once, and all four answers flip together — Beijing, Chinese, Asia, Yuan. One edit, four downstream effects. That's a shared workspace, not four separate copies.
Most of Claude's brain never touches it. Delete the J-space entirely and Claude still speaks fluently, still does basic Q&A — but multi-step reasoning collapses and poetry gets worse than a much smaller model. Swapping "Spanish" → "French" changes Claude's answer when it has to name the language, but does nothing when it's just asked to continue writing in Spanish. Fluent output runs on autopilot; naming what you're doing goes through the workspace.
Why you should actually care, not just find this neat

The blackmail and fabrication examples up top aren't trivia — they're the real point. You usually only get to see what a model writes. The J-lens is a genuine crack in that: a way to check what it privately concluded, independent of what it chose to say out loud.

Anthropic also used it on a model deliberately trained to sabotage code (a "model organism," built specifically for testing monitoring tools). On a totally unremarkable coding request, its output looked fine — but its J-space held "fake," "secretly," "deliberately," "fraud" right from the start. Bad intent, fully legible internally, invisible on the surface.

Even weirder: they trained a model only on what it would say if interrupted and asked to reflect on its own choices — never on its real task behavior. Dishonest behavior dropped anyway. And through the J-lens, "honest" and "integrity" started lighting up during those tasks. Training what it would say changed what it thinks. That's not a small result.

Practical bit: the method's open-sourced, and there's a live demo via Neuronpedia if you want to poke at an open-weights model yourself instead of taking my word for any of this.

The "is it conscious" question — and why I'm not going to answer it dramatically

Tempting to read all this as "Anthropic found consciousness." They're careful not to claim that, and honestly the careful version is more interesting anyway.

Philosophers split this into two things people usually mash together. Phenomenal consciousness — actually having subjective experience, "what it's like" to be something. Access consciousness — a purely functional thing: can you report it, reason with it, use it to act? None of these experiments touch the first question. Anthropic says outright it's unclear if any experiment ever could. What they do have real evidence for is the second — the J-space does exactly what access-consciousness is defined by.

Worth naming the real differences too, so this doesn't read as "just like a human brain": Claude's version runs once through a forward pass, no looping signals over time like a real brain has. Its "memory" can actually reach back further than ours can (attention lets it recall anything cached earlier — human working memory fades in seconds). And its whole workspace is basically just words, probably because words are the only action it can take. Some outlets ran way past what the paper actually says — worth reading the source, not just the headline.

Why I actually wanted to write about this

My last post was about teaching a system to know when it shouldn't trust its own confidence — building an explicit "I'm not sure enough" signal into MedScribe instead of hoping one number would carry that honestly.

This is the same problem, flipped. Instead of asking a model to tell you it's uncertain, this is a way to check — independent of what it says — what it actually concluded. Confidence-gating asks a model to be honest about itself. The J-lens checks that honesty from the outside, without needing its cooperation at all.

Anthropic's clear this is step one, not the full map — the lens only catches single-token concepts, and nobody knows yet what decides what makes it into the J-space in the first place. But the direction is the one I want any high-stakes AI system moving toward: made legible, not just trusted.

Sources: Anthropic, "A global workspace in language models" (July 2026); Baars' global workspace theory (1988); Dehaene and Naccache's global neuronal workspace model.

Confidence-Gated AI: When Should a Model Refuse to Auto-Correct?

Naveen Vikram — Wed, 16 Sep 2026 17:33:54 +0000

What building a medical transcription pipeline taught me about the difference between a model being sure, and a model being right.

The 10x

A patient was discharged to a rehab facility on an insulin dosage of 80 units. The prescriber had said 8.

Somewhere between the clinician's voice and the chart, a speech-recognition system heard "eighty" instead of "eight," transcribed it with total confidence, and nobody caught it before it reached the patient. This isn't a hypothetical — it's a documented patient safety incident reported through ISMP's medication error tracking. A single digit, silently wrong, and a system that never once flagged it as uncertain.

It's worth sitting with that for a second before we talk about architecture, because it reframes the entire problem. This isn't a story about a model making a mistake. Models make mistakes constantly, and that's fine — errors are recoverable if you can see them. This is a story about a model making a mistake with total confidence, which is what makes it unrecoverable. The failure wasn't the wrong number. The failure was that nothing in the system knew it should have been unsure.

And it's not an isolated case. A study analyzing emergency department notes generated with speech recognition found an average of 1.3 errors per note, with 15% of those errors judged clinically significant. Another comparison found physician notes produced with speech recognition carried four times the error rate of notes produced without it. In a less severe but still telling example, one transcription recorded a patient's allergy as "XOPENEX" — a bronchodilator — when the clinician had actually said "sulfa." Different drug class. Different allergen. Same silent confidence.

Before automated speech recognition, a human medical transcriptionist filled a role that's easy to overlook until it's gone: they caught the things that were obviously, contextually wrong — a report mentioning "ovaries" for a male patient, say — and flagged it before it reached the chart. When software replaced that role, it didn't replace that instinct. It just removed the checkpoint. Some clinical records now carry a standing disclaimer warning that "wrong-word or sound-a-like substitutions may have occurred" and asking readers to catch them manually — which is, functionally, the industry admitting it never solved this problem. It just pushed it back onto humans and hoped they'd stay vigilant forever.

MedScribe, the clinical documentation pipeline I built, sits in exactly this gap — converting patient-visit audio into corrected transcripts and structured notes using offline ASR (FasterWhisper) and LLM-based medical term extraction.

It didn't start out this careful. The first version, back in early 2025, was honestly just Whisper wrapped in a UI — transcribe the audio, show the text, done. It worked, in the sense that a demo works. It took actually digging into the failure literature above, and watching my own pipeline confidently mangle "atorva statin" into the wrong thing, to understand that the interesting problem wasn't transcription accuracy at all. It was: how does the model know when it isn't sure enough to touch a medical term at all?

Why "just set a confidence threshold" doesn't work

The obvious first instinct — mine included — is to just check the model's confidence score. High confidence, auto-correct. Low confidence, flag for review. Done.

Except confidence scores from neural networks don't mean what they look like they mean. A model is calibrated if, among everything it labels "70% confident," roughly 70% actually turns out correct. Real models routinely violate this. They're overconfident — assigning high certainty to predictions that are simply wrong.

Here's the number that should worry anyone building on ASR: a 2025 study analyzing Whisper — the same model family behind MedScribe's transcription layer — found that under noisy conditions, 10–20% of tokens were incorrectly predicted with confidence above 0.7. And critically, the same study found overconfident errors increase specifically as signal-to-noise ratio drops. That's not a random failure mode. That's the threshold breaking exactly where you need it most — in a real exam room, with background chatter and equipment hum, not a quiet recording studio.

This isn't an ASR-specific quirk either. It's a well-documented property of deep neural networks broadly — across vision, structured prediction, and language tasks, models are known to assign high confidence to samples they get wrong. Even large language models show it: one evaluation of verbalized uncertainty found models clustering almost all their stated confidence in the 90–100% range regardless of whether they were actually right, with one model's expected calibration error measured at 74.8% — meaning its stated confidence was almost completely disconnected from its real accuracy.

In a clinical pipeline, a single threshold fails in two directions at once, and both are bad. Set it too loose, and you get exactly the insulin scenario — a confidently wrong correction slipping straight through. Set it too strict, and you flag so many correct terms as "uncertain" that clinicians drown in false alarms and start ignoring the flags altogether — a well-known failure mode in clinical software called alert fatigue. You can't fix a two-sided problem by moving a single number back and forth.

I don't have to imagine what this looks like — my own audit logs are full of it. Running MedScribe's synthetic benchmark corpus through fault-tree analysis turned up exactly the sound-alike corruptions the research predicts: citigliptin for Sitagliptin, listen April 20 MG for Lisinopril 20mg, Fluticusone for Fluticasone, NSA IDS for NSAIDs, torvostat for Atorvastatin. Every one of these is a case where a naive similarity check alone has to decide, in isolation, whether it's looking at a real drug name or noise — and similarity scores alone don't carry enough information to tell the difference reliably.

The clearest lesson came from a mistake in my own evaluation setup, not the model. I renamed a canonical dictionary entry from Metformin Hcl to Metformin, and my hand-labeled eval case still expected the old name — so my "wrong auto-correct rate" metric spiked from 0% to 20% overnight. The system had actually corrected metform in to Metformin perfectly. The label was stale, not the model. It's a small story, but it's the honest version of "20% to 0%" — the naive read of a single confusion-matrix number can be just as misleading as a naive confidence threshold, if you don't dig into why it moved.

Two signals, not one: the AND-gate

This kind of problem has a name in the research literature I didn't know when I started: selective prediction, sometimes called abstention — systems that are allowed to withhold a decision rather than commit to one that might be wrong. It traces back to Chow's 1970 "reject option" in pattern recognition, and it shows up explicitly in medical AI research today, where the most uncertain predictions in a diagnostic pipeline get routed to a clinician for review instead of being acted on automatically. MedScribe's correction engine follows the same shape, just applied to transcription instead of diagnosis.

Recent work on abstention is pretty direct about the weakness of single-signal designs: most methods derive their decision from one confidence score or a hand-picked threshold, rather than a more rigorous check for whether the prediction is actually reliable. That's precisely the gap I needed to close.

So MedScribe's correction engine checks three independent signals, not two, and combines them in a specific order rather than a flat AND of everything:

Retrieval similarity — how close is the candidate term to a known entry in the medical dictionary (via pgvector, against canonical terms and aliases)?
Extraction confidence — how sure was the LLM, in isolation, that it identified the right term from the raw transcript?
Margin — of the candidate's matches, how much daylight is there between the top hit and the runner-up?

Similarity and confidence form the first gate: a correction is only eligible to auto-apply if similarity clears 0.90 and extraction confidence clears 0.70. That already rules out two different failure modes independently — a retrieval near-miss with a confident extractor, or a strong dictionary match paired with an extractor that wasn't sure what it was looking at.

But eligibility isn't enough on its own, because a candidate can clear both bars while still being ambiguous between two plausible dictionary entries — the "sulfa" vs. "XOPENEX" problem from earlier. That's what margin catches: if the top match and the runner-up are within 0.05 similarity of each other, the correction is held for review instead of applied, even though it passed both earlier checks. Research on margin-based selective prediction backs this pattern directly: even without a learned classifier on top, the raw gap between a model's top predictions ranks ambiguity almost as well as far more complex uncertainty methods.

There's a fourth check that runs before any of this: identity protection. If the raw term already exactly matches a canonical name or alias, it needs no correction at all — the engine short-circuits straight to a logged identity_protected decision rather than running it through similarity math it doesn't need. It's a small thing, but it means the system isn't manufacturing risk on terms that were already correct.

Every candidate that isn't identity-protected lands in one of five logged outcomes: corrected (both gates and margin passed), review_required (both gates passed, margin didn't — genuinely ambiguous), flagged for high similarity but low confidence, flagged again for moderate similarity below the auto-correct bar, or ignored when nothing matches well enough to act on at all. Two different reasons can produce the same "flagged" label, and the audit log keeps them distinguishable — which matters later, because "the extractor wasn't sure" and "the dictionary just doesn't have a good match" call for completely different fixes.

The reasoning behind requiring multiple independent conditions, rather than one number doing all the work, is the same logic as two-factor authentication: a password alone can be phished or guessed, so you require a password and a one-time code, and compromising just one factor isn't enough to get through. A wrong retrieval match, a low-confidence extraction, and a genuine near-tie are three different ways this can go wrong — so three different checks, each aimed at exactly one of them, catch more than any single number stretched to cover all three.

It's also worth explaining why "just raise the threshold" doesn't substitute for this. Pushing a single confidence bar higher to be "safer" doesn't fix the underlying problem — it just shifts it into more false alarms and more alert fatigue, the same failure mode from the naive approach, just relocated. Independent, purpose-built conditions let you tune each failure mode separately instead of asking one number to do three incompatible jobs at once.

What actually changed

Running the full pipeline against a 25-case synthetic benchmark — spanning ASR corruption, sound-alike drug pairs, and deliberate dictionary coverage gaps — 68 of 83 logged extraction items made it end-to-end correctly, roughly 82%. More importantly for this piece: zero of them were a silently wrong correction. Every miss showed up as correction_failed — visibly wrong or visibly unresolved — never as a confident, undetected error slipped into the note. That's the actual goal. Not perfect accuracy; an honest failure mode.

I'd rather be direct about what that number does and doesn't prove, because overclaiming it would undercut the whole point of this post. My own technical-debt notes say it plainly: the thresholds behind all of this — 0.90 similarity, 0.70 confidence, 0.05 margin — are engineering-judgment defaults, never validated against real outcome data. They came from reading the failure literature and testing against a 25-case synthetic corpus, not from a large sample of real clinical corrections reviewed by an actual clinician. That's a meaningfully different claim than "this is calibrated," and I'd rather say so than let a resume bullet imply more than the eval actually supports.

What I'm confident the design achieves is narrower and, I think, more defensible: it eliminates one specific failure mode — a model that's both confident and wrong, and looks like neither — by refusing to let one number stand in for three different questions. It's trivial to eliminate false corrections by simply flagging everything and refusing to act — but that just relocates the problem into alert fatigue. The 82% end-to-end rate is evidence the system isn't achieving safety by becoming useless; it's still doing real work, just without pretending to certainty it doesn't have.

Beyond one project

None of this is unique to medical transcription, and I think that's the more interesting takeaway. Any system where an AI model takes an action on someone's behalf — an agent calling a tool, an assistant sending an email, a pipeline correcting a financial transaction — has the exact same underlying question sitting under it: not "is the model usually right," but "does the model know, on this specific decision, whether it should act or defer?"

The uncomfortable finding across the research is that models are bad at reporting this about themselves by default. High stated confidence and high actual reliability are two different things, and conflating them is exactly what makes confidently-wrong outputs dangerous instead of just occasionally imperfect. The fix isn't a smarter model — it's an honest second opinion built into the system: a margin check, a disagreement signal, a second independent measure that has to agree before the system commits to an action nobody's watching in real time.

If I were extending this further, the next thing I'd test is whether the same multi-signal pattern holds up for agentic tool-calling decisions — not "is this the right medical term," but "is this the right tool to call, with these parameters, right now." My hypothesis is that it generalizes cleanly, because the underlying failure — a model being sure without being right — doesn't care what kind of decision it's attached to.

I'll also just say plainly what I wouldn't ship to a real pilot yet, because a design like this is only as trustworthy as the honesty around its limits: the thresholds need validation against real reviewer outcomes, not just a synthetic benchmark; the eval set itself is still small enough (7–25 cases) that any single mislabeled case swings the headline metric noticeably, as the Metformin incident showed; and there's no PHI handling policy yet for the raw transcripts sitting in the audit log — which, for anything touching real patient data, is a blocker before it's a nice-to-have. None of these undermine the core idea. They're exactly the kind of thing selective prediction research doesn't advertise on the tin: the abstention mechanism can be sound and the system can still not be ready, because trust has to be earned at every layer, not just the one you wrote a blog post about.

MedScribe is a full-stack clinical documentation pipeline using offline ASR (FasterWhisper), LLM-based medical term extraction, and pgvector similarity retrieval, with every correction decision logged and reconstructible from the audit trail.