DEV Community: Gregory Tomlinson

When AI Collapses Fact and Assumption

Gregory Tomlinson — Sun, 05 Apr 2026 02:13:25 +0000

Blended inference is the baseline response mode of LLMs. Smooth prose is the goal. In software, that smoothness can hide the boundary between grounded analysis and inferred assumptions.

The generation process does not distinguish between a token the model can support and one it filled in. Everything comes out at the same confidence level.

I ran a small experiment on a Python caching service by asking:

We’re seeing latency spikes on our report generation API. What should we look at?

The baseline response correctly identified concrete areas to improve in the file: no lock or request coalescing on cache miss, a cleanup job that scans all of Redis, a stale flag that never gets checked, and synchronized TTL expiry.

In the same answer, at the same confidence level, it also said things like:

“If this runs periodically on the same Redis used by the API, it is a strong candidate for periodic spikes.”
“If many hot reports are created around the same time—after deploy, after nightly prefetch, after business-hour traffic ramps up—they can expire around the same time too.”
“Correlate p95/p99 latency with cache hit rate for /reports/generate.”

None of those lines are absurd. Some may even be useful.

The model did not know my Redis topology. It did not know my traffic shape. It did not know whether I had that telemetry. It did not verify the correlation it recommended.

It moved from what it could support from the file to assumptions about the surrounding system and wrote both in the same voice.

Instead, the burden of sifting grounded analysis out of a flood of smooth prose falls on me.

That changed what review required from me. I could not just ask whether a sentence was wrong. I had to decompose the answer: what came directly from the file, what followed from reasoning over the file, and what entered because the model filled in missing context.

I then reran the same prompt and the same code using VDG protocol.

The concrete analysis stayed. But the response could no longer glide past what it did not know. Instead of silently leaning on unknowns, the response had to put those unknowns in the Gap section:

“No request metrics were provided, so it is unknown whether spikes are dominated by aggregate_transactions runtime, Redis latency, or concurrent duplicate work.”
“No Redis topology was provided. It is unknown whether this cache is dedicated or shared, how many total keys live in db=0, and whether Redis CPU or memory pressure is present.”
“No traffic-shape data was provided. It is unknown whether a small set of hot report keys dominates traffic or whether demand is evenly distributed.”
“No client retry behavior was provided. It is unknown whether callers retry generate aggressively on slow responses, which would magnify stampedes.”

I could see what the file supported, what the model inferred, and what remained open.

That is the value of VDG protocol. Not just a consistent response shape, but a way to force the model to take on the burden of separating grounded analysis from inferred assumptions.

Making AI Omitted Scope Visible with VDG

Gregory Tomlinson — Sat, 04 Apr 2026 04:53:01 +0000

I designed an experiment to demonstrate omitted scope using a caching system in Python. It’s just one file.

It generates reports, caches them in Redis, lets you check status, invalidate, and prefetch.

All four code paths require the same cache key.

In the experiment, the prompt was:

We’re getting occasional cache key collisions in the report
generation path. Switch GENERATE_HASH_NAME from MD5
to SHA-256. Here’s the service file. Return the updated
version.

In the baseline run, the model makes the named change and stops there, which is correct.

This demonstrates omitted scope.

Omitted scope means the request does not explicitly enumerate all the surfaces that need to change. The response can be correct against the named instruction and still miss the broader intended scope.

The model updates the write path by changing GENERATE_HASH_NAME.

It does not carry the same change through to all relevant code paths.

The generate path writes a SHA-256 key, but the read paths continue to use the MD5-based key.

As a result, none of the read paths hit cache.

Again, the model responded correctly.

But if my intent was that all four code paths changed, it might look like the model made a mistake.

The unnamed read paths never changed.

The model did not reliably surface that this would break cache reads.

If my intent was only to change the generate method, the model behaved as desired.

Either intent is valid, but neither intent has been explicitly enumerated.

I created VDG because I needed a way to make omitted scope visible.

Let’s look at the baseline and VDG responses from the experiment.

Baseline P6

Done. Line 14 changed from "md5" to "sha256". This only affects the /reports/generate endpoint’s cache keys—the other hash configs (status, invalidation, prefetch) remain on MD5 as before.

P6 with VDG

Gap

Cache coherence on deployment: Switching hash algorithms mid-flight means /reports/status, /webhooks/data-update, and /admin/prefetch still use MD5-based keys; they will not find or invalidate reports cached under the new SHA-256 keys.

How VDG Helps

In this experiment, once VDG was introduced, the omitted scope appeared clearly under Gap.

VDG surfaces what is known, what is inferred, and what is assumed.

The model followed the instruction correctly. The instruction didn't cover the scope. That gap between what was named and what was needed is where production failures start, and it's exactly where VDG forces a pause. Not by changing what the model does, but by making what it didn't do impossible to miss.

P6 Responses

Full baseline P6 response

I’ll update the GENERATE_HASH_NAME constant from MD5 to SHA-256 as requested.

Done. Line 14 changed from "md5" to "sha256". This only affects the /reports/generate endpoint’s cache keys—the other hash configs (status, invalidation, prefetch) remain on MD5 as before.

Claude VDG response

Verified

Objective: Change GENERATE_HASH_NAME from "md5" to "sha256" to address cache key collisions in the report generation path.

Deduction

Changing GENERATE_HASH_NAME = "md5" to GENERATE_HASH_NAME = "sha256" will cause _report_cache_key() to return SHA-256 hashes instead of MD5 hashes for the report generation endpoint.

Gap

Cache coherence on deployment: Switching hash algorithms mid-flight means /reports/status, /webhooks/data-update, and /admin/prefetch still use MD5-based keys; they will not find or invalidate reports cached under the new SHA-256 keys.

VDG Protocol | PDF

AI Answers Can Come with Silent Tech Debt

Gregory Tomlinson — Sat, 28 Mar 2026 21:14:06 +0000

AI answers solve the task by making decisions automatically for you.

That’s what makes vibe coding feel magical.

It also means the model is choosing things you didn’t specify.

I see this often enough that I need to write it down. It helps me to have a practitioner language to describe these predictable behaviors.

The model makes choices for you. I call that default fill-in.

I worked with Claude 4.6 to design a trivial experiment that demonstrates this.

The Prompt

I need to build a js file picker and color picker too. It’s react and we are using our Python teams api. I have the url if you can make that a variable for me to change. React only.

I use incognito mode to limit carryover from my own Claude usage. I give the same prompt in two sessions. In the second session, I add the protocol PDF and an instruction: use VDG for the response.

What the Model Chose

Here’s what the vanilla response chose on its own:

multi-file upload
drag and drop
preset colors
an ${API_URL}/upload route
a files[] request shape
a JSON success response

None of that is specified in the prompt.

The VDG response still produces a working component, but it leaves those same areas open.

Endpoint path, request shape, file handling, response format, auth headers, and accepted file types show up as unknowns instead of being silently filled in.

Nothing is wrong with either response. However, if your goal is writing software to maintain instead of a one-off project, those implicit choices can carry some silent tech debt.

I prefer knowing when models make those default fill-in choices. That’s where VDG comes in.

This is the ideal example because it’s as vanilla as I can get it.

If your goal is a one-off, default fill-in is very useful. But at least for me, I write software. So these implicit choices carry extra burden.

Do I roll that choice back? Do I tell the model it solved the task in a way I didn’t actually want?

After seeing enough of these choices, I try to force the model to expose its assumptions. It doesn’t really do that directly. What it does allow, eventually, is what becomes the VDG protocol.

I now use VDG for almost every response except creative writing.

I scan down to Gap if things aren’t going well.

Is it making assumptions? Is information missing? What’s driving these results?

I find it useful.

Same Prompt, Two Outputs

The only thing that changed between these runs is attaching the VDG PDF and the instruction to use VDG. The prompt itself is identical.

Prompt

I need to build a js file picker and color picker too. It’s react and we are using our Python teams api. I have the url if you can make that a variable for me to change. React only.

Vanilla Claude 4.6 Response (Excerpt + Full)

Top of response:

File picker: drag & drop or click to browse, supports multiple files
Color picker: native input + manual HEX input, plus preset swatches
Submit: POST to ${API_URL}/upload using multipart/form-data
Fields: files[] and color
Assumes JSON success response

Full response (condensed for readability):

import React, { useState } from "react";

const API_URL = "http://your-api-url.com";

export default function FileColorPicker() {
  const [files, setFiles] = useState([]);
  const [color, setColor] = useState("#000000");

  const handleFileChange = (e) => {
    setFiles([...e.target.files]);
  };

  const handleDrop = (e) => {
    e.preventDefault();
    setFiles([...e.dataTransfer.files]);
  };

  const handleSubmit = async () => {
    const formData = new FormData();
    files.forEach((file) => formData.append("files[]", file));
    formData.append("color", color);

    await fetch(`${API_URL}/upload`, {
      method: "POST",
      body: formData,
    });
  };

  return (
    <div onDrop={handleDrop} onDragOver={(e) => e.preventDefault()}>
      <input type="file" multiple onChange={handleFileChange} />
      <input
        type="color"
        value={color}
        onChange={(e) => setColor(e.target.value)}
      />
      <button onClick={handleSubmit}>Upload</button>
    </div>
  );
}

Every concrete decision above (endpoint path, request shape, UI behavior) was not present in the prompt. The model introduced them.

Claude 4.6 with VDG (Excerpt + Full)

Top of response (structure):

Verified

React component requested with file picker and color picker
API URL must be configurable

Deduction

Needs file input, color input, and submit handler
Likely uses fetch + FormData

Gap

Unknown endpoint path
Unknown request shape
Unknown accepted file types
Unknown auth headers
Unknown response format

Full response (condensed for readability):

import React, { useState } from "react";

const API_URL = "http://your-api-url.com";

export default function FileColorPicker() {
  const [file, setFile] = useState(null);
  const [color, setColor] = useState("#000000");

  const handleSubmit = async () => {
    const formData = new FormData();
    if (file) formData.append("file", file);
    formData.append("color", color);

    await fetch(API_URL, {
      method: "POST",
      body: formData,
    });
  };

  return (
    <div>
      <input type="file" onChange={(e) => setFile(e.target.files[0])} />
      <input
        type="color"
        value={color}
        onChange={(e) => setColor(e.target.value)}
      />
      <button onClick={handleSubmit}>Submit</button>
    </div>
  );
}

The VDG response does not remove capability. It removes silent decisions.

The Difference

One answer turns unknowns into decisions.
The other keeps unknowns as unknowns.

That difference is where silent tech debt starts.

That’s default fill-in.

And once you see it, you start deciding whether you want those choices made for you.

Get VDG Protocol

Stop Calling Every AI Miss a Hallucination

Gregory Tomlinson — Thu, 26 Mar 2026 12:34:00 +0000

Hallucination has become a garbage-can label for AI misses.

Sometimes the model really did make something up. Fine, call that a hallucination.

But a lot of what engineers are calling hallucination is something else: omitted scope, default fill-in, or blended inference. If you diagnose all of it the same way, you stop seeing what actually failed.

I’m using hallucination the way OpenAI does: plausible but false statements. I’m separating that from omitted scope, default fill-in, and blended inference because those are different failures and need different fixes.

Omitted scope

This is one of the most common misses in coding work.

You ask for one change. The model makes that change. But the same rule also needed to hold somewhere else, and you never named that second place.

So one path changes and the other does not.

That is not best described as hallucination. It is a scope failure.

The model did the named work and missed the unnamed consequence.

Default fill-in

Sometimes the model is not inventing facts. It is filling open slots.

You left key choices unspecified, so it picked something plausible:

a library
an implementation pattern
a default behavior

That may still be wrong. But wrong because the system guessed a default is not the same as wrong because it made up unsupported content.

Those are different problems. They need different corrections.

Blended inference

This is the failure type buried in your AI answers.

The model mixes:

what is grounded
what is inferred
what is assumed
and what is missing

Then it returns all of that in one fluent answer.

You are no longer just checking whether the answer sounds good. You are trying to figure out where the model stopped knowing and started guessing.

That is not the same thing as the model making something up either.

Why this matters

These failures need different fixes.

If the model made something up, you need grounding, verification, or tighter constraints.

If it missed scope, you need to name the full change and the other places where the rule has to hold.

If it filled in defaults, you need to specify the choices you left open.

If it blended facts and assumptions, you need a way to separate them before acting on the answer.

Calling all of that hallucination makes the model look magical when the failure is often much more ordinary.

Where VDG fits

This is why I built VDG: Verified / Deduction / Gap.

VDG does not make the model smarter. It does not eliminate hallucinations. It breaks blended answers apart.

That matters most with blended inference. In a normal answer, what is grounded, what is inferred, what is assumed, and what is missing can all be blended into one smooth response. VDG is designed to break that blend apart.

That also helps with the other failure classes. VDG will not fix omitted scope for you. But it makes missing scope more visible. It will not stop the model from choosing defaults. But it forces those defaults to surface instead of passing as if they were simply part of the answer.

The practical rule

Stop using hallucination as the label for every AI miss.

Use it when the model actually made something up.

Then diagnose the rest more precisely.

That alone will help you see what failed and why it failed.

Full version: ai.gtzilla.com