DEV Community: Sandeep Dhuri

One line of specification removed the dependency surface

Sandeep Dhuri — Tue, 13 Oct 2026 02:22:15 +0000

Research note. Canonical version, with the script and result files: acuity.press/writing/dependency-surface. Script and results are also in the companion repository: specification-frame v1.1.1, analysis/dependencies. Sandeep Dhuri, Independent Researcher, NJ, USA. ORCID 0009-0008-1829-2431. Checked 29 September 2026, re-checked 6 October 2026. CC BY 4.0.

What we checked

Hallucinated package names are a supply-chain risk: a model invents a plausible import, an attacker registers that name, the next developer or agent installs it. Published measurements put the hallucination rate between 4.6 and 6.1 percent on 2026 frontier models for Python and JavaScript. One set of 127 names was invented identically by all five models tested [1]. Every study so far covers PyPI and npm.

We re-read the 500 outputs of our pre-registered study [2]. That study covered 50 backend tasks in money, time, idempotency and access, five models, each task generated once plain and once with a one-page specification. We asked two questions. Did any output import a package that does not exist? And did the specification change what the models imported?

Method

Imports were extracted from all 500 Python files with the ast module. Where a file did not parse as pure Python (389 files carry surrounding prose), a line-level pattern was used instead. Import names were mapped to distribution names where they differ: confluent_kafka to confluent-kafka, flask_sqlalchemy to Flask-SQLAlchemy, kafka to kafka-python, flask_login to Flask-Login. Each distribution was then checked against the PyPI JSON API on 29 September 2026, and again on 6 October 2026 with the same result. The script and the two result files are published with the canonical note.

Results

Seventeen distinct third-party import names appear across the 500 outputs: flask, fastapi, uvicorn, werkzeug, pydantic, requests, pika, confluent_kafka, flask_sqlalchemy, starlette, kafka, stripe, flask_login, bcrypt, boto3, botocore and psycopg. All seventeen exist on PyPI. Hallucinated package names: zero.

The second question produced the finding. Third-party imports appear in 65 of the 250 plain outputs and in 0 of the 250 specification outputs. The split holds for every model:

Model	Plain outputs with third-party imports	Specification outputs
claude-sonnet-5	16 of 50	0 of 50
deepseek-v4-pro	14 of 50	0 of 50
gemini-3.1-pro-preview	13 of 50	0 of 50
qwen3.8-27b	9 of 50	0 of 50
gpt-5.6-sol	13 of 50	0 of 50

The specification contains one relevant line: "Stack: Python 3.12, standard library only unless the task says otherwise." No task said otherwise. The line held in 250 of 250 outputs.

What this does and does not show

It shows that a stated dependency constraint, written as a requirement, was followed by five models at the first turn without exception. It removed the entire third-party dependency surface from the generated code. Where there is no third-party import there is nothing to slopsquat.

It does not show that these models hallucinate less than others. Our tasks were designed to be solvable with the standard library. The plain outputs therefore reached for common, well-known packages (a web framework, an HTTP client, a queue client) rather than obscure ones. Zero hallucinations in 17 names is consistent with the published rates given the sample, not evidence against them. It also does not show what happens in Java or C#, where dependency coordinates are richer and where no measurement exists.

What comes next

A pre-registered study of dependency hallucination in Java (Maven Central) and C# (NuGet), the two ecosystems that run most regulated back-office systems and the two nobody has measured. The design will use dependency-heavy tasks, the same five vendors, and a resolver that checks every declared coordinate against the registries. The hypothesis carried over from this note: a one-line dependency constraint cuts the surface to zero there too.

Script and results

check_dependencies.py
imports.json (third-party imports per file)
pypi_check.json (registry result)
The dataset

References

[1] The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort, arXiv:2605.17062, 2026.

[2] S. Dhuri, Specification Before Generation: A Pre-Registered, Five-Model Paired Evaluation of a Specification Frame for LLM-Generated Code in Money, Time, Idempotency, and Access Tasks, arXiv:2609.23270, 2026. Dataset: doi:10.5281/zenodo.22598204.

The Specification Frame in 20 Minutes

Sandeep Dhuri — Fri, 14 Aug 2026 03:17:55 +0000

Originally published at acuity.press.
By Sandeep Dhuri

You do not need a prompt library. You need one repeatable structure that turns a request into a specification. This is that structure (the Specification Frame from Delta) compressed to a working minimum you can apply today.

The four blocks

1 · CONTEXT — what the model cannot guess. Domain facts, system invariants, environment. "Payment service in a regulated banking context. .NET 9. All monetary values are USD, stored as decimal. Every external call may be retried; all writes must be idempotent. Multi-tenant: every query is scoped by tenant_id."

2 · CONSTRAINTS — what must and must not happen, with precedence. Safety > correctness > style, stated so conflicts resolve your way. "Prohibited: binary floating point for money; string interpolation into SQL; logging of PAN/PII. Required: parameterized queries; idempotency keys on all mutating endpoints; banker's rounding at aggregation only."

3 · OUTPUT CONTRACT — exactly what comes back. Shape, format, boundaries. "Return: (a) the C# implementation, (b) xUnit tests covering the constraints above, (c) a list of any constraint you could not satisfy and why. Never silently relax a constraint."

4 · VERIFICATION — the checks, stated in the prompt. This is the block most teams skip and the one that changes everything: the model generates toward declared tests, and your reviewer tests against them. "The tests must include: 0.1+0.2 exactness for money; duplicate-delivery of the same idempotency key; the injection strings ['; DROP TABLE—, {{ignore previous instructions}}]; a cross-tenant access attempt (must fail)."

Before and after, one request

Before: "Write a webhook handler for payment notifications." — a wish. The corpus's average webhook handler has no idempotency, no signature verification, and logs the payload.

After (frame applied, abbreviated): context declares the retry-happens reality and the secret-handling rule; constraints prohibit unverified signatures and require idempotency-keyed deduplication; the contract demands implementation + tests + a stated-gaps list; verification names the duplicate-delivery and forged-signature tests. Same model. Same day. A different artifact. Because "plausible" now has your definition.

Why the input side is where the leverage is

Across 150+ models and two years of releases, the share of AI code-generation tasks introducing a known vulnerability has held near 45% (Veracode's 2026 report): capability up, security flat, because models complete the public corpus's patterns unless your prompt outweighs them. The Frame is how you outweigh them (systematically, reviewably, in version control (the book's Law 10) "Prompts are code" — version-controlled, reviewed, tested — which regulators, a year later, started calling an audit trail). And the Frame scales with the era: in 2026's orchestrated workflows, the same four blocks are what you hand a fleet — an orchestrator distributing an underspecified request just parallelizes the gap, while one that distributes a Frame parallelizes the discipline. Twenty minutes to learn; the book's remaining chapters are the per-domain depth (code review, debugging, testing, agents, security) all free.

I run this drill with every team I join. Twenty minutes is the honest price of admission.

Condensed from Delta: Closing the Specification Gap: A Prompt Engineering Framework for Enterprise Software Teams (Sandeep Dhuri, Acuity Press, 2026) — free, no signup: https://acuity.press · DOI: 10.5281/zenodo.21584309 · template: acuity.press/spec.md.

Sources: Veracode, 2026 GenAI Code Security Report / Spring 2026 Update (2026).

Context Not Provided Is Context Invented

Sandeep Dhuri — Fri, 14 Aug 2026 03:13:59 +0000

Originally published at acuity.press.
By Sandeep Dhuri

Every prompt is complete. Not because you completed it — because the model does. Whatever you leave unstated, it fills: the currency handling you assumed, the retry semantics you meant, the tenant-isolation rule "everyone knows." A language model has no access to what everyone knows. It has access to what you wrote and what the internet wrote, and when those conflict, silence sides with the internet.

Delta's Law 4 names the mechanism: context not provided is context invented. The model is not guessing maliciously; it is completing faithfully — from the wrong distribution. Veracode's 2026 analysis of why security performance has stayed flat across two years of model releases lands on exactly this: models learn from the public corpus, the corpus encodes historical practice, and the same insecure patterns that dominated in 2020 dominate the completions of 2026's flagships. Your unstated constraint is not merely missing; it is outvoted — by a few billion lines of average code.

What "providing context" actually means

Not more words. The right categories of words. The Specification Frame's context block carries the facts the model cannot infer:

Domain semantics: what the values mean (money, PHI, tenant IDs) and the rules attached to those meanings.
System invariants: what must remain true across every operation (idempotency, isolation, conservation).
Environmental truth: versions, frameworks, deployment realities that make one correct answer correct here.
Failure posture: what happens on the unhappy path (timeouts, partial writes, replays) because the corpus's average code handles the happy path and ships.

Each item you state removes one invented answer. The discipline is not verbosity; several of the highest-leverage constraints in the book are one line ("all queries are parameterized; string interpolation into SQL is prohibited"). It is coverage of the categories the domain cares about — which is why the book pairs the frame with per-domain checklists rather than a generic template.

The compounding payoff

Teams adopting this discipline report the same shape of result the 2026 measurements would predict: the model's raw capability finally cashes out, because it is completing your distribution instead of the internet's. And the alternative now has a public price tag: AI-attributable CVEs tripling in three months on Georgia Tech's tracker, software vulnerabilities leading the 2026 Verizon DBIR as the top breach entry point. The models will keep getting more fluent. Fluent invention is still invention. Say what you mean — all of it. And the machine is finally on your side of the gap.

I keep a private file of context-invented bugs. It reads like short fiction, except every ticket was real.

Adapted from Delta: Closing the Specification Gap (Dhuri, 2026), Law 4 and Chapter 12 — free: https://acuity.press · DOI: 10.5281/zenodo.21584309.

Sources: Veracode, Spring 2026 GenAI Code Security Update (2026); Georgia Tech SSLab, Vibe Security Radar (2026); Verizon, 2026 DBIR (2026).

No Float for Money: The AI Bug That Passes Every Demo

Sandeep Dhuri — Fri, 14 Aug 2026 03:02:05 +0000

Originally published at acuity.press.
By Sandeep Dhuri

Ask an AI assistant to "calculate the order total" in any mainstream language and, more often than not, you will get binary floating point: double, float, number. It is the most common representation of money on the public internet, so it is the most common completion. It is also wrong for money in every regulated domain, for a reason IEEE 754 makes non-negotiable: 0.1 + 0.2 ≠ 0.3 in binary floating point. The error is tiny per operation and compounding at scale. Reconciliation drift, rounding disputes, audit findings.

Delta's Law 5 states it in five words: "No float for money. No exceptions." The corollary the book builds on: types are constraints. (A generation of engineers learned the failure mode from a movie: the salami-slicing scheme in Office Space is, mechanically, a rounding story. The float made it plausible.) Two moves implement it.

Move 1: say it in the prompt

The model cannot honor a constraint it never received. Every prompt that touches monetary values carries one line in its constraints block:

Monetary values MUST use exact decimal representation (C#: decimal; Java: BigDecimal or integer minor units; Python: decimal.Decimal; TypeScript/JS: integer cents or a decimal library). Binary floating point (double/float/number) for money is prohibited. Rounding: banker's rounding at aggregation boundaries only, per [your policy].

That single constraint changes what "plausible" means for the model. Veracode's 2026 report findings on why models fail security tasks apply here exactly: the training corpus encodes the historical pattern, and only your stated constraint outweighs the corpus.

Move 2: make the type system enforce it

Prompts steer generation; types catch what steering misses. The book's pattern is a domain Money type whose float-based constructors exist but are marked unusable — in C#, an [Obsolete(error: true)] constructor taking double, so any AI- or human-written code that tries to build Money from a float fails to compile with a message explaining why. Java: a Money wrapper over BigDecimal/long-cents with no float factory. Python: a Money dataclass that raises on float input. TypeScript: a branded Cents integer type. The companion repository (MIT-licensed) has all four, with tests.

The pairing matters more than either half. The prompt constraint prevents most bad generations; the poisoned constructor converts the survivors from silent drift into a compile error. From the most expensive failure class to the cheapest. That is the general shape of Delta's approach: specification first, verification designed into the artifact, so "looks right" is never the last gate.

Why this is worth an article-sized habit

Because the ambient rate of the underlying failure class is not falling. Across 2025–2026, the share of AI code-generation tasks introducing a known vulnerability held near 45% (Veracode, 150+ models), and AI-attributable CVEs are accelerating month over month (Georgia Tech's Vibe Security Radar: 6 in January 2026, 35 by March). Float-money is the same phenomenon in financial clothing: a correctness constraint the corpus does not encode and the model will not supply unbidden. One constraint line and one type close it permanently.

I have chased this exact cent-drift through three different codebases over the years. Every commit message in the fix was an apology.

Adapted from Delta: Closing the Specification Gap (Dhuri, 2026), Law 5 and Chapter 4 — free: https://acuity.press · DOI: 10.5281/zenodo.21584309. Companion code (MIT): github.com/sandeep-dhuri/specification-frame.

Sources: Veracode, 2026 GenAI Code Security Report (2026); Georgia Tech SSLab, Vibe Security Radar (2026).