DEV Community: Chen Yuan

Object Storage Can Replace a Database Under Four Contracts

Chen Yuan — Thu, 10 Sep 2026 15:00:24 +0000

Object storage is not a database. It stores bytes under keys. Yet it can serve as a database-like control-plane store when the workload fits a narrow set of contracts. The useful test is not a yes-or-no replacement claim. It is the set of guarantees the application needs, the ones the storage service supplies, and the ones the application must rebuild. The discussion around Ampbase and Tigris makes the tradeoffs concrete, because the system in question did not start from a desire to be clever. It started from a desire to avoid running a database it did not need, and the work was in rebuilding four guarantees that a database would have provided by default.

The data shape decides the storage engine

The first thing to establish is that object storage is not a general replacement for a database. It is a replacement for a particular shape of data and access. In the Ampbase case, the data partitioned cleanly per organization, the writes were low volume and mostly uncontended, the reads were point lookups on keys the application controlled, and the interesting history was append-only by nature. Those properties are not incidental. They are the reason the design works at all.

Change one of those assumptions and the design changes. Contention on one key turns CAS retries into a possible livelock. A transaction across objects cannot be built from per-object preconditions. Ad-hoc queries turn each new question into a hand-written backfill and index, with application code responsible for correctness.

Storage selection follows the data shape and the questions the application asks. Small, append-only, pre-partitioned, already-aggregated records can fit object storage. High-volume, frequently updated, cross-referenced state is a database workload, and object storage will make you rebuild one badly.

The four guarantees behind the headline

When people reach for a database engine, they often need four behaviors: uniqueness, transactions, indexes, and history. Object storage supplies none of those as database features. Some services do supply two useful primitives: strong read-after-write consistency and conditional writes. The application can build the other behaviors only within the limits of those primitives.

Strong read-after-write consistency means that a successful write is immediately visible to a later read. Amazon S3 documents this behavior for new objects, overwrites, deletes, and listings, and dates its change to December 2020. Without a strong consistency model, a stale read could make an application misjudge uniqueness or compare-and-swap state.

Conditional writes turn compare-and-swap into HTTP preconditions. If-None-Match: * creates only when a key is absent. If-Match: {etag} replaces only the version whose ETag was read. These checks run against the object's latest state within the bucket's consistency model. They provide a concurrency primitive, not a complete transaction system.

These primitives map to the four behaviors in narrow ways. Uniqueness is conditional creation on a content-derived key. Transaction-like mutation is a read, pure computation, and conditional replacement of one object. An index is a key designed around a frequent lookup. History is an append-only key space whose order matches the question being asked.

Each contract has a boundary. Every writer must use conditional creation for uniqueness. A mutation retry must be pure. An index answers only its planned question, and an append-only log records only the events that were successfully appended.

A control plane built from immutable objects

An object-first control plane can use two bucket layers. A global directory bucket records organizations; each organization gets a scoped bucket. Credentials for one customer cannot address another customer's objects, so tenant isolation is an infrastructure boundary rather than an application predicate. There is no WHERE org_id = ? for a developer to forget.

Within a customer bucket, a few key families provide the database-like behavior: a membership key for point lookup, a mutable pointer to the active configuration, immutable version objects, and append-only event objects. The application controls the paths, so these records do not require a general query engine.

Version objects and events are never rewritten, so their audit property comes from the key space rather than from an UPDATE path. Pointers are different: deploying overwrites the active pointer, and rollback moves it to an older version. The history remains because version objects are retained.

The first block shows the key pattern: derive paths from the lookup, then create immutable records conditionally.

import hashlib
import ulid

def membership_key(email: str) -> str:
    digest = hashlib.sha256(email.encode("utf-8")).hexdigest()
    return f"members/{digest}.json"

def version_key(channel_id: str, created_at: str) -> str:
    return f"versions/{channel_id}/{created_at}.pb"

def event_key(org_id: str, event_time: str, event_id: str) -> str:
    return f"events/{org_id}/{event_time}/{event_id}.pb"

The membership key is an index expressed as a path. Hashing the email lets the application compute one GetObject without a listing or secondary index. Version keys use ULIDs; because object listings are lexicographic, a prefix or time range can return history in order. That works only for access patterns designed into the key names. It is not an ORDER BY substitute for arbitrary queries.

What conditional writes can and cannot protect

Conditional writes protect one key from concurrent mutation. They do not coordinate multiple keys or create cross-key atomicity. That boundary is easy to forget when the primitive feels strong.

The second block shows the two forms: create-if-absent and replace-if-unchanged.

def precondition(etag: str) -> tuple:
    if etag == "":
        return None, "*"
    return etag, None

def put_with_precondition(client, bucket: str, key: str,
                          body: bytes, etag: str) -> None:
    if_match, if_none_match = precondition(etag)
    client.put_object(
        Bucket=bucket,
        Key=key,
        Body=body,
        IfMatch=if_match,
        IfNoneMatch=if_none_match,
    )

Concurrent writers to one key receive one success and one precondition error. The loser must re-read: the existing record may be its own retried write or a real conflict. That idempotency check belongs in the application contract.

A mutation function runs again after a conflict, so it must be pure. Side effects inside it, such as an email, counter, or payment call, may happen once per attempt. The compiler will not enforce this; the interface and review must.

Why indexes and history change the design

An index precomputes lookup paths; a unique index also rejects duplicate values. Object-first systems encode both in key names and conditional creation. The tradeoff is fixed access patterns: the application can ask only questions it designed for.

Every access pattern becomes a precomputed key. A new question requires a new key and backfill, which is a hand-written index migration. That is the largest ongoing tax, and it returns with every feature.

An append-only history can resemble event sourcing, but this design stores current state directly and uses events to explain changes. The log is an audit record, not a replay engine.

The third block shows an index record whose path is the lookup and whose conditional creation enforces uniqueness.

import json

def write_index_record(client, bucket: str, key: str,
                       value: dict) -> None:
    payload = json.dumps(value, sort_keys=True).encode("utf-8")
    client.put_object(
        Bucket=bucket,
        Key=key,
        Body=payload,
        IfNoneMatch="*",
    )

Without multi-object transactions, state and event writes can split. If state is written first and the process dies before the event, current state is correct but history has a gap. Every log entry is real, but completeness requires reconciliation.

Deletion is also different. Removing a key erases the distinction between never written and deleted unless a tombstone remains. Keep version objects and move pointers when history matters; storage then grows until a lifecycle policy removes old data.

Multi-region writes need an explicit conflict policy

Multi-region writes are the hardest boundary. A conditional write is evaluated against the state visible in its region. Two regions can therefore accept the same compare-and-swap before replication converges, after which only one version remains current.

A read-after-write check cannot repair this because the lost update happens at the write. The safer rule is to adjudicate each CAS in one primary region and reject dependent writes elsewhere with a client-side guard. Replicas can serve reads, but the write policy must be explicit.

The fourth block makes the retry policy visible: the caller supplies a pure mutation and the loop has a hard attempt limit.

import time

def mutate_with_retry(client, bucket: str, key: str,
                      mutate, max_attempts: int = 5) -> None:
    delay = 0.01
    for attempt in range(max_attempts):
        current, etag = read_object(client, bucket, key)
        new_body = mutate(current)
        try:
            put_with_precondition(client, bucket, key, new_body, etag)
            return
        except PreconditionFailed:
            if attempt == max_attempts - 1:
                raise
            time.sleep(delay)
            delay = min(delay * 2, 0.1)

The example uses a short exponential backoff and five attempts. If contention persists, the caller should surface a conflict rather than loop forever.

Hot keys expose the workload assumption: continuous contention can turn retries into a livelock. Read amplification is the other cost. A directory read per request is repeated by every instance, and listing members before fetching each object creates many round trips. A read-through cache can reduce latency, but it cannot become the source of truth; cold-cache operation must remain correct.

A decision checklist for object-first systems

Before deciding that object storage can replace a database, check the contracts. The data should partition cleanly per tenant, so that isolation is a credential boundary rather than a predicate. Writes should be low volume and mostly uncontended, so that compare-and-swap with retry resolves on the first attempt. Reads should be point lookups on keys you control, so that the access pattern is a key you chose rather than a query you wrote. The interesting history should be append-only, so that immutability is a property of the key space rather than a feature you implemented.

Then check what you are giving up. You are giving up multi-key atomicity, so any invariant that spans objects has to be enforced by the application or abandoned. You are giving up ad-hoc queries, so every new question is a new key and a backfill job. You are giving up the guarantee that the audit log is complete, so if completeness matters you need a reconciliation process. You are giving up simple deletion, so retention has to be an explicit lifecycle policy. And you are giving up single-region simplicity, so multi-region writes need an explicit conflict policy that names the primary region and treats writes elsewhere as errors.

If the workload fits and the contracts are acceptable, object storage can replace a database for that workload. If the workload does not fit, the honest move is to use a database and stop trying to build one from preconditions. The practical rule is this: let the workflow pick the storage engine, and only notice when the data shape has changed enough that the choice should change with it. The original discussion is at https://news.ycombinator.com/item?id=49618450 and the Tigris writeup is at https://www.tigrisdata.com/blog/object-storage-all-need/.

Originally published on Dispatch.

When Safe Browsing Says Clean but Google Ads Says Malware

Chen Yuan — Wed, 09 Sep 2026 15:10:00 +0000

How can a website and a download look clean in several public checks while an advertising account is still rejected for malware? The contradiction is easier to understand when the ad campaign is treated as a delivery system rather than as a single web page.

A current Hacker News discussion points to a useful case. The author of RACE, a native macOS terminal multiplexer written in Rust, reported that a Google Ads account was suspended after spending USD 500 on a campaign. The notice named "Malicious software" and "Compromised Site." The author then checked the website, the download infrastructure, the application, and several public security services, but the appeals still did not reveal which observation caused the decision.

That report does not prove that Google made a mistake, and it does not reveal how the classifier worked. It does show why a publisher needs a record of the whole path from ad impression to first launch. A clean result from one layer is evidence about that layer. It is not a universal verdict about every layer that a visitor may cross.

What the RACE case actually shows

RACE is described as a terminal multiplexer that keeps shell sessions alive across application restarts. Its website is a static Bridgetown site, with downloads hosted separately. The author says it had no custom server-side application at submission time.

The advertising attempt produced a suspension rather than a useful diagnostic. The notice named malicious software and a compromised site. Repeated appeals were rejected, and one path was blocked for a week. The review covered Safe Browsing, Search Console, VirusTotal, signatures, notarization, JavaScript bundles, logs, and different user agents.

One detail matters more than the number of checks. RACE starts and manages background shell processes because process persistence is part of a terminal multiplexer. That behavior can look unusual to a detector even when documented. The report says the application did not inject code into other applications, change browser behavior, or conceal its process activity.

Those are observations from the publisher's investigation, not the platform's internal reason. "The public checks were clean" is defensible; "the campaign was proven safe" is not supported by the same evidence.

Why clean scanners do not clear an ad campaign

The checks in this case answer different questions. Google Safe Browsing examines URLs and warns users in Search and browsers when it detects unsafe sites. Its public status tool can tell a publisher what it reports about a URL at the time of the check. It does not promise that the URL will be treated the same way by an advertising review.

Search Console's Security issues report has a different purpose. Google says the report presents findings when an evaluation determines that a site was hacked or shows behavior that could harm a visitor or their computer. The report can include sample affected URLs, but Google also says the sample may not be complete and that some issues have no example URL. A green result in that report is therefore useful evidence about the report, not a signed clearance for every ad destination and binary.

Apple notarization answers a third question for macOS software. Apple describes its notary service as an automated system that scans for malicious components, checks code-signing issues, and returns a ticket that Gatekeeper can find. Apple also says notarization is not App Review. A valid Developer ID signature and ticket help a user evaluate an application, but they do not determine whether an ad platform accepts a campaign.

Google Ads documentation says reviews may use multiple sources, including the ad, website, accounts, and third-party sources. It does not identify the RACE signal, but it explains why independent reports can disagree. A policy system can inspect more than a URL scanner or notarization service.

The outputs are partial observations. An ad review can consider the relationship between the account, creative, destination, redirects, download, and prior activity. Treating the outputs as interchangeable creates false confidence.

Model the download path as a changing state machine

A publisher should record the path as a sequence of states, not as a screenshot of a landing page. A practical sequence is search query, ad, landing URL, HTML response, script and redirect chain, download host, archive, signed application, first launch, and process or network behavior.

Each state needs a timestamp, URL or artifact identity, observation method, and result. If the download changes while the ad remains unchanged, the reviewed object may no longer be the delivered object. A redirect can also vary by referrer, region, user agent, or cookie. These are engineering possibilities, not claims about the RACE classifier.

A small record type makes the distinction concrete:

from dataclasses import dataclass
from datetime import datetime, timezone

@dataclass(frozen=True)
class Observation:
    stage: str
    url: str
    status: int | None
    artifact_sha256: str | None
    observed_at: str
    method: str


def timestamp() -> str:
    return datetime.now(timezone.utc).isoformat()


record = Observation(
    stage="landing-page",
    url="https://downloads.example.invalid/",
    status=200,
    artifact_sha256=None,
    observed_at=timestamp(),
    method="isolated-http-client",
)

The record does not make a verdict. It preserves context for comparing a later fetch with the earlier one, which matters when an appeal asks which release or response was examined.

A state-machine view prevents a category error. A signed file can be safe at rest while a landing page is compromised, or a clean page can redirect to a different download. A legitimate application can also launch a helper that was not in the original hash.

Verify redirects and artifacts without trusting one layer

The first pass should collect metadata without launching the program. Google’s security guidance warns against opening infected pages directly in a browser and recommends safer methods for examining responses. Use an isolated HTTP client and disposable environment, not a daily workstation.

The redirect chain should be preserved, not reduced to the final URL. Record status codes, location headers, response headers, the request method, and the user agent. Run the same capture from a clean environment when the result matters, then compare the chains rather than choosing the most reassuring one.

from urllib.request import Request, build_opener, HTTPRedirectHandler


class TraceRedirects(HTTPRedirectHandler):
    def __init__(self):
        self.events = []

    def redirect_request(self, request, file, code, message, headers, new_url):
        self.events.append({
            "from": request.full_url,
            "status": code,
            "location": new_url,
        })
        return super().redirect_request(request, file, code, message, headers, new_url)


def fetch_with_trace(url: str) -> tuple[str, list[dict]]:
    tracer = TraceRedirects()
    opener = build_opener(tracer)
    request = Request(url, headers={"User-Agent": "release-audit/1.0"})
    with opener.open(request, timeout=20) as response:
        response.read(4096)
        return response.geturl(), tracer.events

This is a capture tool, not a bypass for an advertising review. Do not rotate identities to evade a policy control. Document the ordinary path a reviewer can reproduce and note whether the destination changes.

The file itself needs an identity. A filename is not enough because a publisher can replace the bytes while keeping the same name. Hash the exact archive that was uploaded, hash the extracted executable when appropriate, and keep the command output with the release record.

from hashlib import sha256
from pathlib import Path


def sha256_file(path: str) -> str:
    digest = sha256()
    with Path(path).open("rb") as stream:
        for block in iter(lambda: stream.read(1024 * 1024), b""):
            digest.update(block)
    return digest.hexdigest()


archive_hash = sha256_file("RACE.dmg")
print({"file": "RACE.dmg", "sha256": archive_hash})

For a macOS release, keep the signing identity, entitlements, notarization result, and hash of the notarized deliverable. If a ticket is stapled after hashing, the reviewed object may differ from the object offered to users.

Build an evidence bundle for a manual review

A useful appeal package is a reproducible bundle, not a pile of screenshots. Identify the ad text, destination, time window, redirect chain, response headers, HTML and script hashes, download hash, signature details, notarization status, and observation environment.

Separate facts from interpretation. An observed field can contain a status code or hash; a hypothesis field can say that a background process may have looked unusual. This keeps an explanation from being mistaken for a platform finding.

import json
from dataclasses import asdict, dataclass
from pathlib import Path


@dataclass
class ReviewBundle:
    ad_id: str
    landing_url: str
    final_url: str
    artifact_sha256: str
    observed: list[dict]
    hypotheses: list[str]


def write_bundle(bundle: ReviewBundle, output: str) -> None:
    Path(output).write_text(
        json.dumps(asdict(bundle), indent=2, sort_keys=True),
        encoding="utf-8",
    )


bundle = ReviewBundle(
    ad_id="campaign-record-2026-09-09",
    landing_url="https://race-term.example.invalid/",
    final_url="https://downloads.example.invalid/RACE.dmg",
    artifact_sha256="record-after-download",
    observed=[{"source": "isolated-http-client", "status": 200}],
    hypotheses=["documented process persistence may need an explicit explanation"],
)
write_bundle(bundle, "review-bundle.json")

The example values are placeholders and must not be submitted as evidence. In a real bundle, do not redact the artifact identity or the time of observation, but do remove cookies, access tokens, and private account data. Keep the original response bytes and logs in a protected archive so the summary can be regenerated.

This format also makes version changes visible. If the binary, redirect chain, or JavaScript bundle changes after an appeal, create a new bundle rather than editing the old one. A reviewer should be able to tell whether the publisher is defending the same object or a later release.

What ad platforms should expose to developers

The RACE report points to a product problem as well as a publisher problem: a generic label leaves no path to test the alleged fault. A review system should expose the category of object that triggered the decision, even if it cannot reveal a sensitive classifier: ad, destination, redirect, download, executable, account context, or third-party report.

It should provide the observation time, sampled URL, response status, artifact hash when available, and environment class. It should distinguish a page warning from an account suspension. If a third-party signal was involved, the publisher needs enough information to identify the asset without seeing private detection rules.

Search Console acknowledges that sample URLs may be incomplete or absent. An appeal interface should show that limit rather than treating a missing sample as proof that nothing was checked, or a clean public report as proof that the ad review had no other input.

That would help both sides: a real compromise would be easier to reproduce and fix, while a false positive would be easier to isolate. The platform could protect its detection logic while giving the publisher a stable object to inspect.

A safer publishing checklist

Freeze the release. Record the exact ad text, destination URL, redirect chain, archive hash, executable hash, signature identity, and notarization result before submitting a campaign.
Capture the normal path with an isolated HTTP client. Preserve redirects, response headers, and final URLs. Do not open an unknown page or program on the workstation used for credentials and daily work.
Check the layers separately. Run Safe Browsing, review Search Console security findings, inspect the download, and verify the platform-specific signing process. Store each result with its timestamp and scope.
Explain unusual behavior plainly. A terminal multiplexer that keeps shell processes alive should document that behavior, its configuration options, and its cleanup path. Documentation does not prove safety, but silence makes a legitimate behavior harder to review.
Submit a factual appeal. State what was observed, what was not observed, and what remains unknown. Do not call a clean scan a certificate, and do not claim to know the classifier's reason without a platform response.
Keep every appeal tied to an immutable bundle. If the binary or destination changes, start a new record, and ask which object and observation must change before a new review can be meaningful.

The RACE case does not establish that Google Ads misclassified a legitimate application. It establishes a narrower engineering lesson: security evidence is scoped to the system that produced it. Safe Browsing, Search Console, Apple notarization, file scanners, and ad review can each observe a different part of the same delivery path. Publishers need reproducible artifacts and a clear separation between observation and theory; platforms need review signals that let a legitimate publisher find the disputed state.

Further reading: the RACE case report, Google Ads policy documentation, Search Console security issues, Apple notarization documentation, and Google Safe Browsing status.

Originally published on Dispatch.

Coding Agents Do Not Test Like Experts: What Agentic Verification Actually Needs

Chen Yuan — Tue, 08 Sep 2026 13:13:03 +0000

A large evaluation of coding agents produced an awkward result. Naming a testing technique did not reliably make the agents use that technique well. In the Zstd implementation study described by Dan Luu, agents were given instructions for TDD, QuickCheck, property-based testing, fuzzing, differential testing, mutation testing, formal methods, and other tools. The default condition, with no extra testing instruction, performed above average. Several named techniques performed worse.

The point is not that testing is useless. The point is that a test library name is not a testing method. Agents can produce tests that compile, run, and miss the bug that matters.

The uncomfortable result is not a missing test command

Coding agents already know that a finished implementation should have tests. They can create a test module, run the project command, repair a failing assertion, and report a green result. That workflow looks like verification from a distance. It often checks only whether the agent's implementation agrees with the agent's own assumptions.

Luu's evaluation makes that gap visible. Agents did not simply ignore every instruction. They often used the requested library or wrote something that resembled the requested technique. The problem was the missing judgment underneath it. A property-based test with a weak generator is still a weak test. A formal proof about an irrelevant invariant does not protect the code path where the defect lives. A differential test that implements the same mistake twice creates agreement, not an oracle.

That distinction matters more as agents take on larger changes. A human reviewer can ask whether a test attacks the risky assumption. An agent tends to optimize for a local signal: a command finished successfully, the assertion passed, or the requested tool appeared somewhere in the diff. The harness must make the stronger question visible.

What the evaluation actually measured

The primary evaluation reused a Rust implementation task for Zstd. It compared 26 prompt conditions, including Default, TDD, QuickCheck, property-based testing, fuzzing, differential testing, mutation testing, Lean 4, Verus, Alloy, and other formal or testing tools. Four additional skills were tested. Correctness came from hidden tests rather than from the tests the agent wrote for itself.

The main graph averaged 80 runs for each condition and effort level and compared medium with higher-effort runs. The article also discusses an IMAP RFC evaluation. The exact ranking should not be treated as a universal league table. The task, model, harness, prompt wording, and available dependencies all affect the result. The author repeatedly warns that the observed differences are noisy and that agents often failed to apply a technique in a meaningful way.

The broad pattern is more useful than the ordering. Default performed above average. At higher effort, fuzzing and property-based conditions did somewhat better than formal methods on average, while the medium-effort picture was mixed. The testing skills recommended by the model underperformed, while a small custom skill did better because it pushed agents toward examining likely mistakes instead of presenting a long tutorial.

This is a study of minimal instructions, not a study of what an expert can achieve with Verus, QuickCheck, or TDD. It asks what happens when an agent receives a technique label and limited guidance. That is close to how many teams deploy skills today, which is why the limitation is also the practical lesson.

Why naming a technique is not using it

QuickCheck was a clear example. The agents generally wrote simple smoke tests, used random inputs that often followed rejection paths, and checked few meaningful properties. In 63 of 160 QuickCheck runs discussed in the study, only one property was checked. The library was present, but the reasoning that makes property-based testing useful was absent.

The formal tools showed a similar split. Verus agents often proved abstract arithmetic facts instead of properties tied to the implementation. Some proofs were effectively of the form A => A: valid, but not useful for finding a Zstd defect. Lean, Alloy, and related conditions also relied heavily on ordinary Rust tests while adding proofs or models that did not reach the risky behavior.

TDD changed the workflow more visibly. Agents wrote about twice as many tests and created failing tests earlier, but the condition still underperformed. More test-first activity did not guarantee better cases. In the four-stream jump-table feature, agents often made all four streams identical. That is a legal fixture for a simple path, but it cannot reveal a bug that appears only when the streams differ.

None of this is an indictment of the tools. It is an indictment of the assumption that a tool name contains its own operating knowledge. An expert knows how to choose generators, construct an oracle, shrink a failure, inspect a counterexample, and decide whether an invariant touches the risk. A prompt that says “use QuickCheck” supplies none of that.

The difference between test presence and test power

Test presence is easy to count. Test power is harder to observe. A suite can contain hundreds of assertions and still leave the important behavior unconstrained.

The Zstd examples show why. Agents noticed that bitstream reversal could be risky, then used palindromic inputs for a relevant test. Reversing a palindrome produces the same sequence, so an encode/decode implementation with the wrong order can pass. The test is related to the feature but powerless against the defect.

The same problem appeared in tests for four Huffman streams. Making every stream identical exercises the shape of the input without exercising the relationship between distinct streams. A fixture that looks realistic can still erase the dimension where the bug occurs.

Differential testing adds another warning. It works when independent implementations receive the same inputs and disagree. In the study, most agents that attempted it did not build two genuinely independent implementations. They wrote the same idea twice, allowing the same wrong assumption to appear in both versions. Agreement then became a false signal of correctness.

A passing test proves only that one execution satisfied one assertion. It does not prove that the assertion represents the specification, that the fixture reaches the risky state, or that the oracle is independent from the code under test. Those are separate questions, and an agent needs a separate artifact for each one.

A verification loop an agent can follow

A useful verification loop starts with a claim instead of a command. The agent states what must remain true, identifies the failure mode that would violate it, chooses an oracle, and records the evidence. A small artifact can make that sequence explicit:

verification:
  claim: decode(encode(data)) equals data for valid inputs
  risk: bitstream order can be reversed in one direction
  targeted_cases:
    - empty input
    - one-symbol input
    - non-palindromic multi-block input
  adversarial_case: distinct values in every stream
  independent_oracle: reference decoder or specification
  required_evidence:
    - test result
    - mutation result
    - failure log when a case is rejected

The claim gives the agent a target that can be reviewed. The targeted cases connect the claim to boundaries and ordinary behavior. The adversarial case tries to falsify the agent's interpretation. The independent oracle prevents the implementation from grading its own homework. The evidence record makes it possible to inspect what the agent actually checked.

The adversarial case should be shaped around the suspected failure, not chosen because it is convenient to serialize. For a stream-order bug, a non-palindromic input is more informative than another randomly generated input that happens to collapse to the same path. For a parser, malformed delimiters and ambiguous boundaries may matter more than another valid example.

A small Rust test can express the difference without pretending to be a complete proof:

#[test]
fn distinct_streams_exercise_ordering() {
    let input = StreamSet::new([
        vec![1, 2, 3],
        vec![4, 7],
        vec![8, 13, 21, 34],
        vec![55],
    ]);

    let encoded = encode(&input);
    let decoded = reference_decode(&encoded);

    assert_eq!(decoded, input);
}

This test is stronger than a palindrome because each stream carries different structure. It still does not prove that the encoder is correct for every valid input. It only supplies one independent, reviewable check aimed at a particular failure mode.

The loop should also have a stop condition. If a mutation survives, the agent must either add a case that kills it or explain why the mutation is outside the specification. A green command with surviving mutations is not a green verification result.

Turn verification into an observable interface

A prompt or skill can nudge an agent, but the environment decides what the agent can learn from failure. The evaluation found that agents could name risky areas such as FSE, Huffman coding, and bit readers without testing them effectively. That is a feedback problem as much as a reasoning problem.

A harness should separate implementation context from verification context where practical. It should provide bounded tools, expose mutation results, preserve failing inputs, and make the independent oracle easy to call. It should also stop runaway work and attach each artifact to a change identifier. The agent then receives information that a longer instruction cannot provide.

Here is a compact interface sketch:

class VerificationHarness:
    def __init__(self, oracle, mutation_tool, fuzzer, threshold):
        self.oracle = oracle
        self.mutation_tool = mutation_tool
        self.fuzzer = fuzzer
        self.threshold = threshold

    def verify(self, implementation, specification):
        claim = propose_claim(specification)
        tests = generate_targeted_tests(implementation, claim)
        if not oracle_checks(self.oracle, tests, specification):
            return Reject("tests do not check the specification")

        mutation_score = self.mutation_tool.run(implementation, tests)
        if mutation_score < self.threshold:
            return Reject("important mutations survived")

        fuzz_result = self.fuzzer.run(implementation, timeout=60)
        if fuzz_result.found_failure:
            return Reject(f"fuzzer found: {fuzz_result.case}")

        return Accept(claim=claim, tests=tests, mutation_score=mutation_score)

The mutation score is not a correctness certificate. It is a pressure signal: tests that merely mirror the implementation should fail to kill useful mutations. Fuzzing is not a certificate either. It adds cases and failure modes that the agent did not choose by hand. The oracle check asks a different question again.

Technique selection should follow the risk and the available oracle. Differential testing fits a protocol with a trusted reference implementation. Property-based testing fits a domain with a clear invariant and a useful generator. Fuzzing fits parsers and state machines with robust crash or semantic oracles. Mutation testing evaluates the tests themselves. Formal methods fit a specification that is precise enough to prove. TDD can structure feedback, but it does not replace hard-case design.

What teams should change first

Start with one high-risk path, not an attempt to impose every testing technique on every task. Write down the independent oracle before asking an agent to implement the path. If the oracle cannot be described, the team is not ready to treat a green agent-generated suite as strong evidence.

Require an adversarial case tied to the likely defect. Do not accept “edge cases included” as a summary. Name the input dimension, state transition, permission boundary, or protocol ambiguity that the case exercises. A reviewer should be able to see why the case could distinguish a correct implementation from a plausible wrong one.

Record verification artifacts with the change. Keep the claim, test rationale, adversarial input, oracle source, mutation result, and preserved failures. This turns review from “the agent says tests pass” into an inspection of the evidence. It also gives a later agent something concrete to revisit when a test fails in production.

Measure escaped defects and holdout failures instead of generated test counts. The study's TDD condition wrote more tests without improving correctness. Counting files or assertions rewards activity, not constraint. Run holdout cases that the agent did not see, mutate the implementation, and compare results against a reference where one exists.

The practical fix is not a longer prompt. It is a verification system that makes weak tests fail visibly, gives the agent an oracle it cannot rewrite, and leaves a reviewer enough evidence to judge the gap between tests that exist and tests that matter.

Originally published on Dispatch.

Rust Trait Objects Are Just a Pointer and a Vtable

Chen Yuan — Sun, 06 Sep 2026 06:51:27 +0000

A trait object in Rust is a fat pointer. That statement is precise, not metaphorical. Every &dyn Trait, Box<dyn Trait>, and *const dyn Trait occupies two machine words: one word holds the address of the concrete value, and the other holds the address of a vtable. The vtable is a static structure generated by the compiler for each concrete type that implements the trait. This layout is the entire mechanism behind dynamic dispatch in Rust.

The Hacker News discussion around the article "Visualizing Rust's Vtables: How dyn Trait Works In Memory" surfaced a common point of confusion: developers coming from C++ expect the vtable pointer to live inside the object itself. In Rust, it does not. The vtable pointer travels with the reference, not with the data. This distinction changes how you reason about memory layout, object size, and API design.

The two words that explain a trait object

A trait object is not a type in the same sense as a struct or an enum. It is a dynamically sized type (DST). The compiler does not know its size at compile time, so you cannot store a dyn Trait directly on the stack or in a struct field without indirection. You must always use a pointer: &dyn Trait, Box<dyn Trait>, Rc<dyn Trait>, or *const dyn Trait.

The pointer itself is the key. A regular pointer in Rust, such as &Circle, is a single machine word containing an address. A trait object pointer, such as &dyn Draw, is two machine words. The first word points to the concrete data. The second word points to the vtable for that concrete type's implementation of the trait.

This two-word representation is why std::mem::size_of::<&dyn Draw>() returns 16 on a 64-bit platform while std::mem::size_of::<&Circle>() returns 8. The data pointer and the vtable pointer together form what the Rust community calls a fat pointer.

The fat pointer does not contain a length, unlike a slice fat pointer which stores a pointer and a length. For trait objects, the metadata is a vtable pointer, not a count. The Pointee trait in the standard library formalises this: for dyn Trait, the Metadata associated type is DynMetadata<dyn Trait>, which is a pointer to the vtable.

What lives in the fat pointer

The first component of a fat pointer is straightforward: it points to the concrete value. That value can be anywhere—stack, heap, or static memory. The pointer is raw address information, nothing more.

The second component points to a vtable. The vtable is a static structure, typically placed in read-only data segments, that contains the function pointers for all the trait's methods, plus some additional metadata required by the language.

The vtable layout is an implementation detail of the Rust compiler, not a stable language guarantee. However, the general structure is well understood from compiler source and documentation. A vtable begins with three entries:

A pointer to the drop_in_place implementation for the concrete type. This is how Box<dyn Trait> knows how to drop the value when it goes out of scope.
The size of the concrete type, in bytes.
The alignment of the concrete type, in bytes.

After these three header entries, the vtable contains one function pointer for each method defined in the trait, in declaration order. For traits with supertraits, the vtable may also contain pointers to the supertrait vtables to support trait upcasting coercion.

Each concrete type gets its own vtable for each trait it implements. If a type implements both Draw and Clone, the compiler generates a Draw vtable and a separate Clone vtable. When you coerce a &Circle to &dyn Draw, the fat pointer uses the Draw vtable. When you coerce the same &Circle to &dyn Clone, it uses the Clone vtable.

All instances of the same concrete type share the same vtable. The vtable is per-(type, trait) pair, not per-instance.

Reading a vtable without guessing

The standard library documentation exposes std::ptr::DynMetadata as the conceptual type for trait-object metadata, but the pointer-metadata APIs are still nightly-only in the current stable documentation. That distinction matters: the type is useful vocabulary for understanding what the compiler carries, but it is not a portable stable inspection interface. On stable Rust, std::mem::size_of::<&dyn Draw>() and ordinary coercions let you reason about the representation without extracting metadata.

use std::mem;

trait Speak {
    fn speak(&self) -> &str;
}

struct Dog;
impl Speak for Dog {
    fn speak(&self) -> &str { "woof" }
}

fn main() {
    let dog = Dog;
    let trait_obj: &dyn Speak = &dog;

    println!("trait object reference: {} bytes", mem::size_of_val(&trait_obj));
    println!("concrete reference: {} bytes", mem::size_of::<&Dog>());
}

On a 64-bit target, the first line reports 16 and the second reports 8. The example measures the reference representation, not the size of the Dog value. It is stable and safe, but it does not expose the vtable pointer or its function slots.

Reading the actual function pointers from a vtable is not a stable operation. The vtable layout is not guaranteed across Rust versions or target platforms. Any attempt to read vtable slots by offset is a debugging experiment, not production code. Even comparing vtable pointer values is unreliable: the compiler may duplicate or merge vtables during code generation.

Static dispatch and dynamic dispatch side by side

Static dispatch, using generics, produces a separate monomorphised copy of the function for each concrete type. The compiler knows exactly which draw implementation to call at each call site. There is no vtable, no fat pointer, and no runtime overhead for dispatch.

trait Draw {
    fn draw(&self) -> &str;
}

struct Circle;
struct Square;

impl Draw for Circle {
    fn draw(&self) -> &str { "Circle" }
}

impl Draw for Square {
    fn draw(&self) -> &str { "Square" }
}

// Static dispatch: monomorphised per type.
fn draw_statically<T: Draw>(shape: &T) -> &str {
    shape.draw()
}

fn main() {
    let c = Circle;
    let s = Square;
    println!("{}", draw_statically(&c));
    println!("{}", draw_statically(&s));
}

The compiler generates draw_statically::<Circle> and draw_statically::<Square> as separate functions. Each function contains a direct call to the corresponding draw implementation.

Dynamic dispatch, using dyn Trait, uses a single function that accepts a fat pointer and dispatches through the vtable.

fn draw_dynamically(shape: &dyn Draw) -> &str {
    shape.draw()
}

fn main() {
    let c = Circle;
    let s = Square;
    let shapes: [&dyn Draw; 2] = [&c, &s];
    for shape in &shapes {
        println!("{}", draw_dynamically(*shape));
    }
}

The draw_dynamically function is not monomorphised. There is one copy of the function. Each call to shape.draw() loads the vtable pointer from the fat pointer, finds the method entry for Draw::draw, and calls that function with the data pointer as the self argument.

Dynamic dispatch adds an indirect call and can limit inlining compared with a statically known concrete type. The actual cost depends on the surrounding code, cache behaviour, and whether the compiler can remove or specialise the abstraction. It should be measured in the workload that matters, not reduced to a universal number of loads.

The choice between static and dynamic dispatch is a trade-off between optimisation opportunities and flexibility. Static dispatch enables inlining and devirtualisation when the compiler can see the concrete type, but requires compile-time knowledge of the participating types. Dynamic dispatch allows heterogeneous collections and runtime polymorphism, but gives up some of those opportunities.

Object safety is an API boundary

Not every trait can be used as a trait object. The compiler enforces a set of rules called object safety, also referred to in recent compiler versions as "dyn compatibility". These rules determine whether a trait can be converted to a dyn Trait type.

A trait is dyn-compatible if its methods can be called through a trait object. The most important rules are:

Methods cannot have generic type parameters unless the method is restricted with where Self: Sized. A vtable entry must represent one callable signature, while a generic method would require monomorphisation for each type argument.
Methods cannot return Self by value unless they are similarly restricted. A trait object does not know the erased concrete size needed for such a return.
A method cannot use Self in a position that requires the erased type's size, such as a by-value argument, unless the method is restricted with where Self: Sized.
Associated constants are not dyn-compatible. Associated types are allowed when the trait object specifies the associated type, as in dyn Iterator<Item = u8>.

// Dyn-compatible.
trait Safe {
    fn do_something(&self);
    fn do_another(&self) -> i32;
}

// Not dyn-compatible: generic method.
trait UnsafeGeneric {
    fn generic<T>(&self, x: T);
}

// Not dyn-compatible: associated constant.
trait UnsafeConst {
    const KIND: &'static str;
}

// Dyn-compatible when the object supplies Output.
trait HasOutput {
    type Output;
    fn get(&self) -> Self::Output;
}
// A usable object type can be written as dyn HasOutput<Output = i32>.

The compiler will refuse to coerce a value to &dyn UnsafeGeneric or Box<dyn UnsafeConst>. A trait with an associated type is not automatically rejected; the object type may need to bind that type before it can be used.

The object safety rules are an API boundary, not a performance optimisation. They exist because the compiler cannot generate one callable vtable entry for operations that still require compile-time type information. A vtable does not choose generic method instantiations or provide storage for a size-varying return. An associated type can work when the trait object fixes its value in the object type.

The Rust reference explicitly defines object safety in terms of what operations can be performed on a trait object. A trait object permits "late binding" of methods, dispatched using vtables. If a method cannot be dispatched through a vtable, the trait is not object safe.

A small memory probe in Rust

You can observe the fat pointer layout directly using std::mem::transmute or by casting to a raw pointer and reading the two words. This is an unsafe debugging technique, not a stable API. The following example demonstrates the layout on a 64-bit platform.

use std::mem;

trait Identify {
    fn id(&self) -> &str;
}

struct Item(u64);

impl Identify for Item {
    fn id(&self) -> &str { "Item" }
}

fn main() {
    let item = Item(42);
    let trait_ptr: &dyn Identify = &item;

    // A fat pointer is two words.
    println!("size of &dyn Identify: {} bytes", mem::size_of::<&dyn Identify>());
    // On 64-bit: 16 bytes.

    // Unsafe inspection: read the two words of the fat pointer.
    let (data_ptr, vtable_ptr): (*const (), *const ()) = unsafe {
        mem::transmute(trait_ptr)
    };

    println!("data pointer: {:p}", data_ptr);
    println!("vtable pointer: {:p}", vtable_ptr);

    // The data pointer points to the `item` variable.
    // The vtable pointer points to a static vtable for Item + Identify.
}

The transmute call is unsafe because it bypasses Rust's type system. The language does not promise a stable vtable ABI or a stable ordering for private vtable slots. This code is useful when investigating a particular compiler build, but it is not a general FFI contract. The old core::raw::TraitObject pattern should not be treated as a supported public API.

The pointer-metadata APIs describe the same split more explicitly, but they are still nightly-only in the current standard-library documentation. A nightly experiment can reconstruct a raw trait-object pointer from matching parts:

#![feature(ptr_metadata)]

use std::ptr;

fn inspect_again(trait_ptr: *const dyn Identify) {
    let metadata = ptr::metadata(trait_ptr);
    let data = trait_ptr as *const ();
    let reconstructed: *const dyn Identify = ptr::from_raw_parts(data, metadata);

    // The metadata must belong to the same erased trait-object type.
    assert_eq!(trait_ptr as *const (), reconstructed as *const ());
}

The functions are safe in the narrow sense that constructing the raw pointer does not dereference it, but a caller must still ensure that the data pointer and metadata describe a valid Identify object before dereferencing the result. That is a compiler-experiment boundary, not a stable application interface.

Rules for using dyn Trait deliberately

Use dyn Trait when you need runtime polymorphism in a collection or when the set of types is not known at compile time. A Vec<Box<dyn Draw>> can hold circles, squares, and triangles together. A Vec<Circle> cannot.

Avoid dyn Trait when the concrete type is known at compile time and you do not need heterogeneous storage. Static dispatch through generics produces faster code and enables inlining.

When designing a trait that you intend to use as a trait object, ensure object safety from the start. Generic methods and Self-returning methods are the most common blockers. If you need a trait to be object safe and also support generic methods, consider splitting the trait into an object-safe core and a separate generic extension trait.

The vtable pointer in a fat pointer is not a stable ABI. Different Rust versions or different compiler flags may change the vtable layout. Code that relies on a specific vtable offset is fragile and should be confined to unsafe debugging or specialised crates that track compiler internals.

The DynMetadata API provides stable access to size and alignment. Use it when you need to know the size of the concrete type behind a trait object. Do not attempt to read the function pointer slots directly in production code.

The cost of dynamic dispatch is two additional memory indirections per method call. In performance-critical code paths, consider using an enum over a fixed set of types instead of dyn Trait. An enum allows static dispatch through pattern matching while still supporting heterogeneous values.

Object safety is a property of the trait definition, not the implementation. You cannot make an object-unsafe trait object-safe by changing the implementation. The trait's method signatures must satisfy the object safety rules for any dyn Trait to exist. If you control the trait, design it to be object safe. If you do not control the trait, use generics or wrapper types.

The fat pointer representation of trait objects is one of Rust's most elegant design decisions. It separates the mechanism of polymorphism from the data itself, keeps objects small, and enables zero-cost abstractions where static dispatch is sufficient. A trait object is just a pointer and a vtable. Everything else follows from that fact.

Originally published on Dispatch.

Inside the Sandbox Is Still Code Execution: Chromium CVE-2026-85046 for Agent Operators

Chen Yuan — Sat, 05 Sep 2026 14:17:14 +0000

Is a renderer compromise only a containment failure, or is it already a breach of the agent's trust boundary? CVE-2026-85046 makes that contrast concrete. The bug is a type confusion flaw in Chromium's V8 engine. Google patched it in Chrome 152.0.7977.82 after confirming it was exploited in the wild. The NVD description is narrower than the headlines that followed: a crafted HTML page can run arbitrary code inside the sandbox. That sentence is easy to read as a relief. For an AI agent that browses the open web with a logged-in session, it is the opposite.

A sandbox is a containment layer. It limits what a renderer process may do to the rest of the machine. A trust boundary is the set of actions the agent is allowed to take as the user. Those two lines are not the same. In-sandbox code execution crosses the second line without needing to cross the first.

What the advisory actually says

The NVD entry for CVE-2026-85046 names the component, the class of bug, the fixed build, and the impact. The component is V8. The class is type confusion, catalogued as CWE-843. The fixed build is Chrome 152.0.7977.82. The impact is remote code execution inside the sandbox from a crafted HTML page. Chromium rated the issue High. Public writeups credit Salvatore Gulizia, also known as Serotav, with the report.

Type confusion is not a logic error in a web API. V8 represents JavaScript values with hidden classes and typed slots. A successful exploit creates a value under one type and later forces the engine to read those slots as another type. Once that happens, attacker-controlled JavaScript can treat a small integer as a pointer, or a pointer as a length. From there the usual memory-corruption path is available inside the renderer: arbitrary read, arbitrary write, then control of the instruction pointer in that process.

The important qualifier is the process, not the machine. Chromium splits work across a browser process and one or more renderers. Site isolation puts different sites in different renderers when it can. V8 lives in the renderer, so the bug starts in the process that already has the page's DOM and the cookies that page is allowed to touch.

"Inside the sandbox" describes the operating-system jail around that renderer. It does not mean the page, the session, or the agent remains honest.

In-sandbox code execution is not a sandbox escape

The sandbox is an operating-system policy around the renderer: restricted tokens and job objects on Windows, seatbelt on macOS, namespaces and seccomp on Linux. Those controls are meant to stop a hostile renderer from opening arbitrary files or talking to the rest of the desktop as an equal.

CVE-2026-85046 does not remove that jail. Code that runs after the type confusion still runs as the renderer. It still sees the page and still speaks to the browser process through the same IPC channels a legitimate renderer uses.

That is a different bug class from a sandbox escape. The same Chrome release also fixed CVE-2026-85050, an out-of-bounds write in WebGL on Android that NVD describes as code execution outside the sandbox. A full browser exploit chain often wants both stages: first a renderer primitive, then a second bug that breaks the jail. CVE-2026-85046 is the first stage. CVE-2026-85050, on the platforms where it applies, is the second.

Treating "no sandbox escape" as "no incident" collapses the two stages into one. For Chrome's security team the distinction is real. For an agent whose job is to click and read as the user, the first stage already steals the product. The attacker does not need a new host process if the existing renderer can drive the same session.

Why agent browsers inherit the same bug

Playwright, Puppeteer, Chrome DevTools Protocol clients, and extension-based automation do not ship a private JavaScript engine. They start or attach to Chromium. The V8 in that binary is the V8 in that Chrome build. If the build is older than 152.0.7977.82, the advisory applies.

There are at least three common ways an agent gets a browser, and they do not update together.

The first is the user's daily Chrome. Auto-update may patch it on a schedule the operator does not control. That is the binary people mean when they say "I already updated Chrome."

The second is Chrome for Testing, or the Chromium revision Playwright downloads into a cache directory. Teams pin that binary so CI is reproducible. Pinning last month's revision is a stability choice. It is also a decision to keep last month's V8.

The third is attaching to an already running profile. Hermes-style "use my logged-in browser" automation, Playwright's channel attach, and some CDP scripts skip the pinned testing binary and speak to the browser that already has cookies. That path inherits whatever build the user actually launched. It also inherits the session.

A patched desktop Chrome therefore does not repair a pinned testing binary. A patched testing binary does not repair an old Electron shell. An updated library package does not replace a browser downloaded once and left in a cache. The bug lives in the executable that runs V8, not in the Python that called browser.new_page().

import re
import subprocess
from pathlib import Path

MIN_CHROME = (152, 0, 7977, 82)


def parse_version(text: str):
    match = re.search(r"(\d+)\.(\d+)\.(\d+)\.(\d+)", text)
    if not match:
        return None
    return tuple(int(part) for part in match.groups())


def chromium_version(browser_path: Path):
    result = subprocess.run(
        [str(browser_path), "--version"],
        capture_output=True,
        text=True,
        check=False,
    )
    if result.returncode != 0:
        return None
    return parse_version(result.stdout or result.stderr)


def assert_patched(browser_path: Path):
    version = chromium_version(browser_path)
    if version is None:
        raise SystemExit("refusing to start: browser version is unknown")
    if version < MIN_CHROME:
        raise SystemExit(
            "refusing to start: Chromium %s is older than 152.0.7977.82"
            % ".".join(str(part) for part in version)
        )
    return version

A pinned browser is a security decision

Pinning a browser revision is not a missing update. It is a choice to prefer a known binary over a moving one. That choice is reasonable for a test suite that only opens fixtures on localhost. It is a poor default for an agent that follows links, opens email HTML, or visits sites chosen by a model.

Playwright's install step downloads the Chromium revision that matches the package. Upgrading the Python package without re-running the install leaves the old binary in the cache. Docker tags have the same shape: a tag that was current in August is not a patch for a September V8 advisory.

The other pinning trap is flags. --no-sandbox is still common in containers because it makes Chrome start under root. That flag does not create CVE-2026-85046, but it removes the containment layer the advisory still relies on. --disable-web-security and --disable-site-isolation-trials widen what a hostile renderer can reach after it already runs code. A version gate that ignores flags is a version gate that lies.

A production agent should treat the binary path, the four-part version, the launch flags, and the profile as one reviewable object. If any field is unknown, the process should not open a URL. Unknown is not "probably fine." Unknown is "the check did not run."

import sys
from pathlib import Path

# Reuses assert_patched() from the previous listing.
FORBIDDEN_FLAGS = {
    "--no-sandbox",
    "--disable-web-security",
    "--disable-site-isolation-trials",
}


def assert_launch_safe(browser_path: Path, launch_flags: list[str]) -> None:
    version = assert_patched(browser_path)
    forbidden = FORBIDDEN_FLAGS.intersection(launch_flags)
    if forbidden:
        raise SystemExit("refusing to start: forbidden flags %s" % sorted(forbidden))
    print("chromium %s accepted" % ".".join(str(part) for part in version), file=sys.stderr)


def open_untrusted_url(page, url: str, browser_path: Path, launch_flags: list[str]) -> None:
    assert_launch_safe(browser_path, launch_flags)
    if not url.startswith("https://") and not url.startswith("http://"):
        raise SystemExit("refusing to open a non-http URL")
    page.goto(url)

Containment is not the same as trust

After a renderer compromise, the attacker has the same view the page had, plus the ability to ignore the page's own JavaScript and talk to the renderer internals. That is enough to read cookies the renderer is allowed to send, fill forms, click buttons, inspect the DOM, and intercept traffic that already terminates in that process.

For a personal browsing agent the blast radius is the user's session. The agent was going to use that session anyway. The attacker now uses it first. Anything the renderer could already send, including cookies and filled forms, is in scope.

For an attached profile the blast radius is larger still. The automation did not create a throwaway login. It borrowed the user's real Chrome. In-sandbox code execution then sits next to every other tab in that profile's renderer topology. Site isolation helps, but it is not a promise that a hostile origin cannot be opened, and it is not a promise that the agent's privileged tab is in a different process from the attacker's page.

The sandbox still does useful work. It is why a renderer bug is not automatically a kernel bug, and why "steal the session" and "install a host implant" should stay different tickets. The mistake is to file the first one as "not an incident because the sandbox held."

An agent is a program whose feature is to act. The renderer is where that acting happens. Control of the renderer is control of the feature. "Inside the sandbox" answers a different question: whether that control also reaches the rest of the disk as the host user.

What a version check should prove

A useful check answers four questions with evidence, not with a changelog.

Which executable started? The path must be the path the agent will launch or attach to, not a google-chrome --version from some other install on the same machine.

Which four-part version did that executable print? Compare it as a tuple against 152.0.7977.82. String equality against a single build is too brittle, and a major-only check is too loose.

Which flags were actually passed? Parse the launch argument list and fail closed on --no-sandbox, --disable-web-security, and --disable-site-isolation-trials.

Which profile is attached? A throwaway user-data-dir is a different decision from the person's daily profile. If the policy forbids attaching to the real profile, a running Chrome with that profile is a failed start.

browser_policy:
  min_version: "152.0.7977.82"
  attach_user_profile: false
  allowed_launch_flags:
    - "--headless=new"
    - "--disable-dev-shm-usage"
  disallowed_launch_flags:
    - "--no-sandbox"
    - "--disable-web-security"
    - "--disable-site-isolation-trials"
  on_unknown_version: block
  on_forbidden_flag: block

The policy is small on purpose. It does not detect exploits. It only refuses to start a browser whose version or flags the operator has not accepted, including when CI reuses an old cache.

The boundary you can actually review

You cannot review whether a random page contains a V8 exploit. You can review which Chromium binary the agent starts, whether that binary is at least 152.0.7977.82, whether the sandbox flags are intact, and whether the session is a disposable profile or the user's own Chrome.

Those four facts belong in the same place as the rest of the agent's contract: a file, a startup assertion, and an exit status. They do not belong in a wiki reminder to "keep Chrome updated." The desktop auto-updater and the agent cache are different programs. Treating them as one program is how a patched laptop still runs a vulnerable renderer for the model.

The sandbox remains worth keeping. It is the reason CVE-2026-85046 is not described as code execution on the host. Keep the flag that enables it. Keep site isolation.

Then keep the other line in view. An agent that browses untrusted HTML hands a renderer a session and asks it to act. Code execution inside that renderer is already enough to steal cookies, drive the page, and act as the logged-in user. Chrome's security team will still care whether the jail held. The operator should care whether the agent still belonged to them. For this class of product, "inside the sandbox" is already a total loss of the session the agent was built to use.

Originally published on Dispatch.

The Skill Is the Product: Making AI Coding Agents Obey Real Repositories

Chen Yuan — Thu, 03 Sep 2026 11:33:07 +0000

A Hacker News post on September 2, 2026 linked Matt Pocock's public repository, Skills For Real Engineers. The repository is not a new agent runtime. It is a collection of small Markdown skills taken from the author's .agents directory, with an installer for putting selected files into a project. The discussion around the post made the useful question sharper: which instructions belong in a general repository guide, and which deserve their own reusable workflow?

That distinction matters once an agent edits a real codebase. A short prompt can suggest an approach. A repository guide can describe local rules. A skill can package a repeatable operation with its trigger, supporting files, and expected checkpoints. The last item is valuable only when the operation has a boundary that a human can inspect. Otherwise, a skill is just a longer prompt with a more impressive filename.

The repository is part of the agent

An agent does not arrive at a repository with the same assumptions as its author. It must learn the project’s names, tests, issue tracker, architecture, and definition of done. The README for Pocock’s collection describes this problem directly: agents are often asked to discover a project’s vocabulary as they work, then spend many words restating concepts that the project could have named once.

The collection’s CONTEXT.md example shows the practical fix. A project-specific phrase can replace a long explanation of a domain event. That is not decoration. It gives the agent a stable search term for functions, files, tests, and design notes. The repository becomes part of the skill’s input rather than a blank directory that the agent may interpret differently on every run.

This does not mean that every local rule belongs in a skill. A global rule such as “run the formatter before committing” fits an AGENTS.md or CLAUDE.md file. A workflow such as “turn an ambiguous request into a spec, then create tickets” has a trigger, a sequence, and a visible result. That workflow is a better skill candidate. The test is simple: could a teammate invoke it by name and know what artifact should exist when it finishes?

What a skill actually contains

Pocock’s repository uses ordinary Markdown files, grouped into directories such as skills/engineering and skills/productivity. Its README separates user-invoked skills from model-invoked skills. User-invoked entries include flows such as /grill-me, /to-spec, and /implement. Model-invoked entries include reusable practices such as test-driven development, bug diagnosis, code review, and codebase design. The distinction is about who may start the flow, not about whether one kind of file is magically more intelligent.

The repository also shows why a skill is different from a README. A README explains a collection and links to its members. A skill file describes the work to perform, the questions to ask, the files to consult, and the checks to complete. The surrounding repository supplies scripts, templates, reference documents, and installation metadata where needed. Together, these parts form an operational package.

It is useful to keep the contract small. A skill needs a trigger, inputs, ordered actions, and an observable verification rule. Those fields do not claim to be the schema of Pocock’s repository. They are a local convention for teams that want to make a file-based workflow easier to review and test.

name: add-health-route
trigger:
  - add a health route
inputs:
  handler_file:
    type: path
    required: true
  route_path:
    type: string
    required: true
    allowed_prefix: "/"
steps:
  - action: append_if_missing
    path: "{{ handler_file }}"
    content: |
      @app.get("{{ route_path }}")
      def health():
          return {"status": "ok"}
verify:
  - action: contains
    path: "{{ handler_file }}"
    text: "@app.get(\"{{ route_path }}\")"

The example is deliberately boring. It names one file, accepts two values, performs one mutation, and checks one result. It does not promise to understand a whole web framework. That modesty makes review possible. A maintainer can ask whether the route belongs in that file, whether the decorator is correct, and whether the verification is strong enough before the agent runs it.

Start with a narrow contract

A broad instruction such as “improve this service” gives an agent too many legal interpretations. A narrow contract removes choices that do not belong to the task. It can state the permitted directories, the files that must already exist, the command that may run, and the result that proves completion. When an assumption is false, the skill should stop before mutation and report the missing condition.

Narrow does not mean trivial. Pocock’s collection includes workflows for grilling through a plan, writing a test-driven change, diagnosing a difficult bug, and reviewing a diff. Each workflow is substantial, but each has a recognizable seam. tdd is about a red-green-refactor loop. diagnosing-bugs is about building a feedback loop around one hard bug. code-review examines a bounded diff against standards and the originating specification.

The contract should also say what the skill will not do. A route skill should not silently redesign authentication. A ticket skill should not publish a specification to an issue tracker without making the destination explicit. A test skill should not claim success because a file changed. Negative boundaries protect the repository from enthusiastic automation.

The input contract is where safety begins. Reject absolute paths if the operation is meant to stay inside the repository. Reject a missing required value. Restrict enumerated options. Resolve defaults before the first write. These checks are cheap, and they turn an ambiguous request into a visible failure instead of a guess.

Turn the contract into executable checks

Written instructions explain intent, but an exit status or an exact file check gives the agent something firmer to observe. The verification need not be elaborate. It can assert that a generated file exists, a required string appears once, a test command exits successfully, or a diff stays inside an allowed directory.

A useful check tests the outcome rather than the action. “The append step ran” is weak evidence. “The route decorator appears in the intended file, the test passes, and no file outside the allowlist changed” is stronger. The more consequential the operation, the more the verification should inspect the external state that matters.

The same rule applies to skills copied from a public collection. Installation is not verification. The README for mattpocock/skills offers two paths: a Claude Code plugin or the skills.sh installer, which can copy selected editable files into a repository. After installation, a team still needs to confirm that the chosen files are in the expected directory, that their triggers are reachable in the chosen agent, and that project-specific paths and terminology have been adapted. A copied workflow is a starting point, not proof that it fits.

A small runner can make the local convention testable without introducing a framework. The important details are path confinement, input validation, bounded commands, and structured results. The runner below treats the YAML contract as data. It never passes a rendered command through a shell.

from pathlib import Path
import subprocess
import yaml

class SkillRunner:
    def __init__(self, contract_path: Path, repo_path: Path):
        self.contract_path = contract_path
        self.repo_path = repo_path.resolve()
        with contract_path.open(encoding="utf-8") as stream:
            self.contract = yaml.safe_load(stream)

    def _inputs(self, provided):
        values = {}
        for name, spec in self.contract.get("inputs", {}).items():
            if name in provided:
                value = provided[name]
            elif "default" in spec:
                value = spec["default"]
            elif spec.get("required"):
                raise ValueError(f"missing required input: {name}")
            else:
                continue
            allowed_prefix = spec.get("allowed_prefix")
            if allowed_prefix and not str(value).startswith(allowed_prefix):
                raise ValueError(f"invalid value for input: {name}")
            values[name] = str(value)
        return values

    def _render(self, text, values):
        for name, value in values.items():
            text = text.replace("{{ " + name + " }}", value)
        return text

    def _path(self, value):
        candidate = (self.repo_path / value).resolve()
        if candidate != self.repo_path and self.repo_path not in candidate.parents:
            raise ValueError("path leaves repository")
        return candidate

    def run(self, provided):
        values = self._inputs(provided)
        for step in self.contract.get("steps", []):
            if step["action"] == "append_if_missing":
                path = self._path(self._render(step["path"], values))
                path.parent.mkdir(parents=True, exist_ok=True)
                content = self._render(step["content"], values)
                current = path.read_text(encoding="utf-8") if path.exists() else ""
                if content not in current:
                    path.write_text(current + content, encoding="utf-8")
            elif step["action"] == "command":
                argv = [self._render(str(item), values) for item in step["argv"]]
                completed = subprocess.run(
                    argv, cwd=self.repo_path, capture_output=True,
                    text=True, timeout=int(step.get("timeout", 30)),
                    check=False,
                )
                if completed.returncode:
                    return {"ok": False, "phase": "step", "returncode": completed.returncode,
                            "stderr": completed.stderr}
            else:
                raise ValueError(f"unsupported action: {step['action']}")

        for check in self.contract.get("verify", []):
            path = self._path(self._render(check["path"], values))
            expected = self._render(check["text"], values)
            if check["action"] != "contains" or expected not in path.read_text(encoding="utf-8"):
                return {"ok": False, "phase": "verify", "check": check.get("name", "unnamed")}
        return {"ok": True, "phase": "verify"}

This is not a general agent framework. It supports two file-level actions and one command action, and it returns early on a failed command or check. That is enough to demonstrate the property a skill needs: the procedure is explicit, and the result is not inferred from a confident paragraph.

Keep context local and explicit

Context should be close to the workflow that consumes it. A repository-wide guide can define language, package manager, test command, and protected directories. A skill can point to the small set of files needed for its operation. A reference document can explain a domain term without forcing every task to reread the whole project.

Pocock’s README makes this separation concrete through CONTEXT.md, ADRs, and skills that update or use them. The point is not to put every fact into one enormous instruction file. Large instruction files become difficult to audit, and agents may spend attention on rules unrelated to the current change. A pointer to a focused document is often better than a duplicate explanation.

Local context also improves portability. The public collection can be installed as editable files, but the README warns that a team should choose one installation philosophy rather than installing the same skills twice through different mechanisms. After installation, replace generic assumptions with the project’s actual issue tracker, documentation location, labels, and vocabulary. A skill copied unchanged is not automatically a team process.

The context boundary should be visible in the contract. List the files the skill may read. List the paths it may write. Name commands and give them time limits. If a workflow needs a human decision, produce a proposal and stop at that seam. Do not hide a product decision inside a filesystem helper.

Test failure paths, not demos

The happy path proves that the example was arranged correctly. Failure tests prove that the boundary exists. For a file-writing skill, test a missing required input, a path outside the repository, an unsupported option, a command timeout, and a verification mismatch. The assertion should cover both the returned status and the absence of an unsafe mutation.

A failure result should be useful to the next action. “Failed” is not enough. Include the phase, the named check, the return code, or the rejected input. Avoid copying an entire environment into the result; a short diagnostic is easier for a human and an agent to inspect.

from pathlib import Path
import yaml
from runner import SkillRunner

def make_contract(path: Path, expected: str = "@app.get(\"/health\")"):
    contract = {
        "inputs": {"handler_file": {"type": "path", "required": True}},
        "steps": [{
            "action": "append_if_missing",
            "path": "{{ handler_file }}",
            "content": '@app.get("/health")
',
        }],
        "verify": [{
            "name": "route",
            "action": "contains",
            "path": "{{ handler_file }}",
            "text": expected,
        }],
    }
    path.write_text(yaml.safe_dump(contract), encoding="utf-8")

def test_successful_run(tmp_path):
    contract = tmp_path / "skill.yaml"
    make_contract(contract)
    result = SkillRunner(contract, tmp_path).run({"handler_file": "app/routes.py"})
    assert result == {"ok": True, "phase": "verify"}
    assert '@app.get("/health")' in (tmp_path / "app/routes.py").read_text()

def test_failed_verification_is_reported(tmp_path):
    contract = tmp_path / "skill.yaml"
    make_contract(contract, expected="marker-that-is-not-written")
    result = SkillRunner(contract, tmp_path).run({"handler_file": "app/routes.py"})
    assert result["ok"] is False
    assert result["phase"] == "verify"
    assert result["check"] == "route"

The second test is intentionally unglamorous. The write happens, but the contract still reports failure because the stated outcome is absent. That distinction prevents a runner from confusing “the step executed” with “the task succeeded.” In a real repository, the verification would usually call the project’s existing tests rather than inventing a second test system.

A small Python skill runner

The runner’s most important design choice is what it refuses to do. It does not interpret free-form prose as a command. It does not accept a path that escapes the repository. It does not turn a missing input into a guessed default unless the contract declares that default. It does not report success until every verification entry passes.

The command action is still a sharp tool. A fixed argument list is safer than a string assembled for a shell, but it can still delete data or contact an external service. Give the skill an allowlist of commands, use a short timeout, capture output, and run it in a temporary checkout when the operation is experimental. For deployment, billing, credentials, or irreversible migrations, make the human approval an explicit boundary instead of pretending that a local exit code proves the remote state.

This is also where existing engineering skills help. Pocock’s collection treats test-driven development, diagnosis, architecture, and code review as separate disciplines that can be combined around a change. A local runner should not reproduce all of those practices. It should invoke the project’s established tools and verify the seam that belongs to the skill.

Where this approach stops helping

A skill contract is a good fit when the task has a stable trigger, a small input surface, and a deterministic check. It is a poor fit for work whose correctness depends on an unresolved product decision, a broad architectural tradeoff, or a remote system that cannot be inspected from the repository. In those cases, the skill should gather evidence, produce a proposal, and stop before the decision point.

The Hacker News comments on Pocock’s repository raise a fair boundary question: some practices could fit in a general AGENTS.md or CLAUDE.md, while a skill is more useful for a named workflow, a project-specific script, unusual domain knowledge, or a sequence that benefits from explicit gates. That is a better rule than treating every useful paragraph as a new command.

The practical recommendation is to start with one narrow skill that touches a known seam. Define its inputs and refusal conditions. Reuse the repository’s own tests. Add failure cases before sharing the file. If the skill grows a long list of exceptions, stop adding branches and move the complex reasoning into a normal tool or a human review step. The product is the bounded, inspectable workflow. The agent is the caller that applies it to the repository in front of it.

Originally published on Dispatch.

Why uv Now Deduplicates Every Wheel in Its Cache (and How It Does It)

Chen Yuan — Mon, 31 Aug 2026 15:51:22 +0000

The Problem: Duplicate Wheels in the Cache

A developer working on a monorepo with two Python projects—a FastAPI web service and a data pipeline—notices something odd. Each project has a requirements.txt file listing around 200 dependencies. When they run uv pip install for the first project, uv downloads and extracts 200 wheels into its global cache. Then they run uv pip install for the second project, and uv does the same thing again.

But here's the kicker: these two projects share roughly 150 of the same dependencies. pydantic==2.10.4, httpx==0.27.2, typing-extensions==4.12.2—the list goes on. Yet when they check their disk usage, they find that uv has stored 400 copies of extracted wheels on disk. The same typing-extensions wheel, byte-for-byte identical, exists twice in the cache. They're effectively paying double the storage cost for a dependency they use in both projects.

This isn't an edge case. In any Python environment with multiple projects, virtual environments, or CI pipelines, the same packages get pulled down and extracted repeatedly. The waste compounds quickly. A monorepo with 10 microservices might store 10 copies of numpy, 10 copies of pandas, and 10 copies of scikit-learn—each taking up hundreds of megabytes.

Why does this happen? Prior to the content-addressed cache feature, uv's cache stored extracted wheels in archive-v0 under randomly generated IDs. If the same wheel was reached through different cache entries—for example, fetched from two different indexes—uv stored duplicate copies because it identified extracts by their source rather than their contents.

This identity-based approach is simple and fast—you can look up a package by name and version instantly. But it leads to significant redundancy when many projects share the same dependencies.

Before: How uv's Cache Organized Data

Before the content-addressed cache features landed, uv's cache had a two-tier structure. Downloaded wheels lived in the wheels-v0 or wheels-v1 bucket, keyed by a combination of the package name, version, and a hash of the wheel file itself. Extracted wheels lived in archive-v0, and each extract got its own randomly generated directory ID.

You can inspect your current uv cache location with:

uv cache dir

On Linux and macOS, this typically outputs something like $HOME/.cache/uv. On Windows, it's %LOCALAPPDATA%\uv\cache.

If you peek inside the archive-v0 directory, you'll see a bunch of seemingly random directory names:

ls $(uv cache dir)/archive-v0/

On a real system, this might show entries like:

0VSKRv7KPsU6OVxKhRKyA/
3vlR8U66f3qKdzSrxDtyR/
gekN10oQ_Vp-g2TxEYqsD/

Each of these opaque directories contains the full extracted contents of a single wheel. The IDs are randomly generated—they carry no relationship to the files inside. This means that if you install the same wheel from PyPI and then from a local directory, uv treats them as different cache entries and stores two copies of the extracted files.

To see just how much duplication exists, you can use tools like du to measure the total size of the cache and compare it against the unique file count:

du -sh $(uv cache dir)
ls -R $(uv cache dir)/archive-v0/ | wc -l

For a project with many dependencies, the archive-v0 directory can contain tens of thousands of files. A significant portion of these are duplicates—the same .py files, the same compiled extensions, the same native libraries, stored over and over again.

Content-Addressable Storage: A Brief Primer

Content-addressable storage (CAS) is a pattern where data is stored and retrieved by its content hash rather than by a name or location. Instead of asking for "file typing-extensions-4.12.2.dist-info," you ask for "the file whose BLAKE3 hash is b7a3...." If two files have identical contents, they produce identical hashes, and the storage system returns the same object for both requests.

This concept is not new. Git uses content-addressable storage for its object database. Every blob, tree, and commit is identified by its SHA-1 hash, which is why Git can detect duplicate files across your repository and only store them once. Docker layer caching works on a similar principle—if two images share the same base layer, they can reuse it without storing a second copy. npm deduplicates tarballs in its cache using content hashes, ensuring that if you install the same package in different projects, you only download it once.

The key insight is that content-addressable storage turns the problem of "how do I avoid storing the same data twice" into a simple lookup: compute the hash of the content, check if an object with that hash already exists, and if so, reuse it. The hard part is managing the links—making sure that when you ask for the contents of a package, the system knows which hashed objects to assemble, and when a package is removed, the system can safely garbage-collect unreferenced objects.

How uv's Content-Addressed Cache Works

uv's content-addressed cache evolution happened in two stages. The foundational work landed in PR #19693, which introduced content-based directory hashes for entire extracted wheels. The idea was simple: instead of storing an extracted wheel under a randomly generated ID, compute a hash of the directory's entire contents, and store it under that hash. If two different wheels extract to the exact same directory tree, they'll produce the same hash, and uv can deduplicate them.

The more significant optimization came in PR #21327, which moved deduplication from the wheel level to the file level. In this newer implementation, every file extracted from a wheel is stored individually in a files-v0 bucket, keyed by its BLAKE3 content hash. The original directory structure is preserved through hardlinks—the archive-v0 directory contains hardlinks pointing to the actual file objects in files-v0.

Here's how the flow works:

When uv needs to extract a wheel, it iterates through every file in the archive.
For each file, it computes a BLAKE3 hash of the file's contents.
It checks if an object with that hash already exists in the files-v0 bucket.
If the object exists, uv creates a hardlink from the extracted wheel's location to the existing object.
If the object doesn't exist, uv writes the file to the files-v0 bucket under its hash, then creates a hardlink to it.
The extracted wheel directory in archive-v0 now contains hardlinks to shared file objects, with each object stored exactly once on disk.

This approach preserves the existing installation pipeline—the wheel installation step still reads files from their expected paths in archive-v0—but the underlying storage is now fully deduplicated.

The PR benchmarks showed impressive savings. On a local machine with a typical Python cache, the file-level deduplication saved 545.2 MiB of disk space, or about 10% of the total cache. The optimization covered all payload files in the cache—not just executables and native libraries, but every .py, .so, .dll, and data file.

The performance impact was also carefully measured. Cold installs—where the cache is empty and files must be written for the first time—showed a slowdown of less than 4% on median. Warm installs, where the cache is already populated, showed no statistically significant difference. The team found this tradeoff worthwhile—a small performance hit on cold installs for a noticeable reduction in disk usage.

The feature uses BLAKE3 as its hashing algorithm, which is known for its speed and parallelizability, making it well-suited for hashing many files during wheel extraction.

Enabling the Preview Feature

The content-addressed cache features are currently behind preview flags. To use them, you need to explicitly opt in. uv provides several ways to enable preview features.

First, you can enable the feature via the --preview-features flag on individual commands:

uv pip install --preview-features content-addressed-cache -r requirements.txt

To enable it for all commands in a session, you can use the UV_PREVIEW_FEATURES environment variable:

export UV_PREVIEW_FEATURES=content-addressed-cache
uv pip install -r requirements.txt

You can also enable preview features in your uv.toml file or under [tool.uv] in your pyproject.toml:

[tool.uv]
preview-features = ["content-addressed-cache"]

For the physical space accounting feature—which accounts for hardlinks when reporting cache size—you need the cache-physical-space preview feature as well. You can enable multiple features simultaneously:

uv cache size --preview-features content-addressed-cache,cache-physical-space

Or set the environment variable:

export UV_PREVIEW_FEATURES=content-addressed-cache,cache-physical-space

Once enabled, uv will start using content-addressed storage for new wheel extractions. Existing cache entries will remain in their current format—uv doesn't automatically migrate old cache data to the new structure. You can either let new installs populate the new format gradually, or you can clear your cache and start fresh:

uv cache clean

Note that preview features are subject to change. uv explicitly warns that they may be modified or removed without notice, and they should not be relied upon in production environments until stabilized.

Measuring the Savings

To measure how much space your cache is actually using on disk—accounting for hardlinks—you need the cache-physical-space preview feature. Without it, uv cache size reports the apparent size of the cache, which counts the same file multiple times if it's hardlinked in different locations. With the physical space accounting, the command reports the actual disk usage.

Run the following command to see your cache's physical space usage:

uv cache size --preview-features content-addressed-cache,cache-physical-space --human

The --human flag displays the size in a human-readable format (e.g., 1.2 GiB instead of raw bytes).

The actual savings you'll see depend on your workload. The benchmark in PR #21327 showed a 545.2 MiB reduction on a local machine, which was about 10% of the total cache. However, that benchmark was on a single machine with a specific set of packages. In practice, the savings could be higher or lower depending on how much overlap exists between your dependencies.

For workloads with many large packages that share common files—like multiple machine learning frameworks that bundle the same native libraries—the savings could be significantly more. The per-file selection table from the PR shows the pattern: selecting only executables and native libraries saves 275.7 MiB across 3,336 files, while selecting all payload files saves 545.2 MiB across 134,222 files.

To see the breakdown, you can also use system-level tools to compare the apparent size of the extracted tree against the actual disk usage of the object store:

du -sb $(uv cache dir)/archive-v0   # Apparent size: every hardlink counts full
du -s  $(uv cache dir)/files-v0     # Disk usage: each unique object once

The difference between these numbers represents the space saved by file-level deduplication.

What This Means for CI and Local Development

For local development, the primary benefit is reduced disk usage. If you work on many Python projects simultaneously, a smaller cache means you're less likely to run out of disk space on your development machine. It also means less SSD wear—hardlinking does not write new data to disk, so the cache's write amplification is lower.

For CI pipelines, the impact is more nuanced. CI systems often cache the ~/.cache/uv directory between runs to avoid re-downloading packages. But the cache's archive-v0 directory contains many small files that take time to compress and decompress. With file-level deduplication, the cache contains fewer distinct file objects in files-v0, which may improve compression ratios and reduce cache transfer times.

However, the best practice for CI is still to cache only the downloaded wheels (wheels-v1) and not the extracted archives (archive-v0), since the archive can be regenerated from the wheels. The content-addressed cache doesn't change this recommendation—if anything, it makes it more important to be selective about what gets cached.

For monorepo setups, where many projects share dependencies, the benefits compound. A single typing-extensions wheel extracted once into files-v0 and hardlinked across 10 project caches would save 9 copies of the extracted files. Over hundreds of shared dependencies across dozens of projects, the savings can add up quickly.

It's worth noting that this feature is still in preview. Both PR #19693 (wheel-level dedup, merged 2026-08-25) and PR #21327 (file-level dedup, merged 2026-08-31) are in the development branch and have been released under the content-addressed-cache preview flag in uv 0.12.7. Early adopters should expect some rough edges, especially around cross-platform compatibility. The initial implementation was validated on Linux with the ext4 filesystem. Supporting macOS, Windows, and other filesystems may require additional work. The team has considered edge cases like reflink support on copy-on-write filesystems (e.g., Btrfs, ZFS, APFS), but these features are still under development.

Originally published on Dispatch.

A Stand-In Is Not a Backup: Designing for a Full Cloud Outage

Chen Yuan — Sat, 29 Aug 2026 11:57:42 +0000

A full cloud outage is a different failure

A cloud provider can fail in a way that leaves machines running but makes the system hard to operate. Deployment APIs, identity, DNS, service discovery, and queues may be unreachable. A backup image is not much help if the team cannot provision it or obtain credentials. This differs from an Availability Zone (AZ) failure, where the rest of the platform remains under normal control.

A stand-in asks what must remain usable when the primary control plane is unavailable. It is a separate service for a few critical operations, with enough cached state to make bounded decisions. It is a second operating path with a smaller promise, not a restore point waiting to be rehydrated.

The design brief changes. Multi-AZ and multi-region deployments reduce different risks; a stand-in needs its own failure domain and activation path. It may use another cloud or a separately managed environment, but it cannot depend on the primary control plane. AWS's reliability guidance frames disaster recovery around recovery objectives and architecture. A stand-in adds a stricter question: can the recovery system operate while the primary is unreachable?

What Monzo Stand-in gets right

Monzo's engineering post describes Stand-in as separate backup banking infrastructure. The Primary Platform runs on Amazon Web Services, while Stand-in runs on Google Cloud Platform. The platforms have independent Kubernetes clusters and separate services. Stand-in exposes limited endpoints and a simplified app experience for card spending, cash withdrawal, bank transfers, balance and transaction views, and freezing or unfreezing cards.

The important choice is separation. Monzo's post says that common operations, including card payment processing, are implemented independently in the two platforms. Stand-in is therefore not a copy of the core banking platform with another database connection. Different code reduces the chance that one software defect, deployment mistake, or assumption will take down both paths.

The design also explains why strong consistency was not applied to every replicated value. If both platforms had to acknowledge every write, losing either could block updates. Monzo keeps the primary as the system of record and treats Stand-in decisions as temporary. Disaster recovery becomes a product boundary: preserve a few useful actions, and define what happens to everything else.

Independence beats perfect replication

A common response to cloud-provider risk is to replicate the entire deployment in a second region or cloud. A full replica can fit some recovery objectives, but it is expensive and complex. It also reproduces shared assumptions: a bad identity policy, Kubernetes upgrade, or shared DNS or release system can affect both copies.

A stand-in chooses independence over feature parity. Instead of duplicating every service, the team identifies the smallest set of operations that must survive. For a bank, that may be payment processing and transfers. For a SaaS platform, it may be read-only access to critical data or the ability to start a failover process manually. The stand-in is a deliberately limited system, not a second edition of the whole product.

Traditional active-passive recovery can still be useful, but its contract is to promote the same service with the same data model and APIs. A stand-in accepts a different API and reduced functionality when that separates failure modes. The goal is a working path while the primary is unreachable, not an invisible handoff.

Sync only the state you can tolerate

During a network partition, a design cannot promise both strong consistency and availability for the same write. A stand-in needs a bounded stale view and a policy for each action. Monzo's post describes non-blocking, eventually consistent replication, immutable data copied from the Primary Platform, separate state created while Stand-in is active, and a durable queue of effects for recovery.

Monzo says its minimal state includes balances, limited transaction history, card and account details, pots, and payees. It monitors replication lag rather than pretending the view is perfect. A smaller system should carry a monotonic version or source timestamp, reject older events, and alert when freshness crosses policy.

A cached balance may be stale. A read-only view can show its last known value, while an irreversible write may require a conservative decline or review queue. Each accepted action needs an idempotency key and durable record so recovery can distinguish a new operation from a replay.

Keep the emergency surface small

A small emergency surface is easier to reason about, test, and operate under stress. Every additional service adds dependencies, configuration, and another way for activation to fail.

This is a deliberate architectural constraint. For a payments system, the minimum useful surface might authorize a transaction from cached state and record the decision for later reconciliation. For an e-commerce platform, it might serve product listings from a static cache and let authenticated users view their order history. The exact boundary comes from the customer promise, not from the list of services already running in the primary platform.

The stand-in should not include administrative or analytics features or expose the full primary API. Every endpoint is a potential defect; every synchronized table is a freshness and recovery problem. A boring service with a clear refusal mode is more useful than a broad service that fails halfway through an operation.

Design the switch as a product

Activating a stand-in is not a simple DNS cutover. It changes the customer experience, the write path, and the recovery plan. Treat the switch as a product feature rather than an operational afterthought.

The activation control should be explicit and reversible. An operator or independent health check can request the transition, but the stand-in should refuse activation when its cache is empty, sync lag is outside policy, or its dependencies are unhealthy. Record the decision and its reason in an audit log the primary cannot erase.

Observability needs the same independence. Logs, metrics, traces, and alert delivery cannot depend only on the primary monitoring stack. Customer communication also belongs in the design: users may see fewer features or delayed transactions, and support staff need a visible state that explains why.

During Stand-in operation, some values are stale, some writes are provisional, and some requests must be rejected. Making those rules visible is safer than routing traffic to a hidden copy and asking downstream services to infer what happened.

Test the path before you need it

A stand-in that has never exercised its activation path is an unverified assumption. Test promotion, degraded behavior, and return to the primary. A small matrix can run regularly:

Scenario	Property to verify
Stale snapshot	The service rejects or limits an unsafe write and still serves the actions allowed by policy.
Duplicate event	Replaying an event does not double-apply an effect or change a settled decision.
Failed promotion	A missing dependency leaves the service in a known state and does not advertise partial availability.
Primary recovery	The stand-in drains new writes, emits its durable effects, and returns control without losing the audit trail.

Fault injection should remove the dependencies the design claims not to need: block primary DNS, revoke primary identity permissions, disable a control-plane API, and interrupt sync. Run the tests against the real activation mechanism, not a test-only flag. Kubernetes readiness and liveness probes help with process health, but they do not prove fresh business state; that condition needs its own test and alert.

A small reference implementation

These examples illustrate a stand-in service. They are not production-ready and have not been tested in an outage. The service is a transaction authorizer with a bounded sync queue, explicit transitions, and a recovery record.

Start with a queue that fails loudly when it is full. A bounded queue that silently evicts old snapshots can turn a known outage into an unknown data gap.

import asyncio
from dataclasses import dataclass
from enum import Enum
from queue import Empty, Full, Queue

class StandInState(str, Enum):
    IDLE = "idle"
    ACTIVE = "active"
    DRAINING = "draining"
    FAILED = "failed"

@dataclass(frozen=True)
class AccountSnapshot:
    account_id: str
    balance: int
    frozen: bool
    version: int

class SyncQueue:
    def __init__(self, max_items: int = 10_000) -> None:
        self._items: Queue[AccountSnapshot] = Queue(maxsize=max_items)

    def put(self, snapshot: AccountSnapshot) -> None:
        try:
            self._items.put_nowait(snapshot)
        except Full as exc:
            raise RuntimeError("sync queue is full") from exc

    def drain(self) -> list[AccountSnapshot]:
        snapshots = []
        while True:
            try:
                snapshots.append(self._items.get_nowait())
            except Empty:
                return snapshots

The queue rejects input when full, giving the sync worker an observable failure instead of discarding the oldest state. The snapshot version lets the consumer ignore delayed events that would move an account backward.

The service owns its transitions and applies snapshots only when their versions advance:

class StandInService:
    _allowed = {
        StandInState.IDLE: {StandInState.ACTIVE, StandInState.FAILED},
        StandInState.ACTIVE: {StandInState.DRAINING, StandInState.FAILED},
        StandInState.DRAINING: {StandInState.IDLE, StandInState.FAILED},
        StandInState.FAILED: {StandInState.IDLE},
    }

    def __init__(self) -> None:
        self._state = StandInState.IDLE
        self._accounts: dict[str, AccountSnapshot] = {}
        self._lock = asyncio.Lock()

    @property
    def state(self) -> StandInState:
        return self._state

    async def transition(self, target: StandInState) -> None:
        async with self._lock:
            if target not in self._allowed[self._state]:
                raise RuntimeError(f"invalid transition: {self._state} -> {target}")
            if target is StandInState.ACTIVE and not self._accounts:
                raise RuntimeError("cannot activate without cached state")
            self._state = target

    async def apply_snapshot(self, snapshot: AccountSnapshot) -> bool:
        async with self._lock:
            current = self._accounts.get(snapshot.account_id)
            if current and snapshot.version <= current.version:
                return False
            self._accounts[snapshot.account_id] = snapshot
            return True

    async def authorize_transaction(
        self,
        account_id: str,
        amount: int,
    ) -> tuple[bool, str]:
        if amount <= 0:
            return False, "invalid_amount"
        if self._state is not StandInState.ACTIVE:
            return False, "stand_in_not_active"

        async with self._lock:
            account = self._accounts.get(account_id)
            if not account:
                return False, "account_not_cached"
            if account.frozen:
                return False, "account_frozen"
            if account.balance < amount:
                return False, "insufficient_balance_cached"
            return True, "approved"

The authorizer never calls the primary. The caller still needs an idempotency key and a reservation or ledger rule; this example only shows the availability decision. A real policy may decline more aggressively when the snapshot is old.

The activation check should combine primary reachability with stand-in readiness. The probes must use paths that remain available when the primary control plane is not.

from dataclasses import dataclass
from typing import Awaitable, Callable

@dataclass(frozen=True)
class Health:
    primary_up: bool
    cache_ready: bool
    sync_lag_ok: bool

async def decide_state(
    service: StandInService,
    probe_primary: Callable[[], Awaitable[bool]],
    cache_ready: Callable[[], bool],
    sync_lag_ok: Callable[[], bool],
) -> StandInState:
    health = Health(
        primary_up=await probe_primary(),
        cache_ready=cache_ready(),
        sync_lag_ok=sync_lag_ok(),
    )
    if health.primary_up:
        if service.state is StandInState.ACTIVE:
            await service.transition(StandInState.DRAINING)
        return service.state
    if (
        service.state is StandInState.IDLE
        and health.cache_ready
        and health.sync_lag_ok
    ):
        await service.transition(StandInState.ACTIVE)
    return service.state

The function refuses promotion when the cache or freshness check is unhealthy. Recovery moves to DRAINING first, giving the write path a visible stopping point.

After recovery, reconciliation should classify records before applying anything:

from dataclasses import dataclass

@dataclass(frozen=True)
class Advice:
    transaction_id: str
    account_id: str
    amount: int

def classify_advices(
    primary_ids: set[str],
    already_applied: set[str],
    advices: list[Advice],
) -> tuple[list[Advice], list[Advice], list[Advice]]:
    seen = set(already_applied)
    duplicates, review = [], []
    for advice in advices:
        if advice.transaction_id in seen or advice.transaction_id in primary_ids:
            duplicates.append(advice)
        else:
            review.append(advice)
            seen.add(advice.transaction_id)
    return [], duplicates, review

async def begin_rollback(service: StandInService) -> None:
    await service.transition(StandInState.DRAINING)

This deliberately auto-applies nothing. A duplicate is recorded, and an unseen advice goes to review or a separately designed idempotent step. A delayed payment is visible; a double-applied payment is harder to undo.

The trade-off is deliberate

Designing for a full cloud outage means choosing which failure a service must tolerate. A full replica optimizes feature parity, but it can carry the same software, identity assumptions, and operational dependencies into the recovery environment. A stand-in optimizes independence and accepts a smaller promise.

That promise has costs. Users see fewer features, some state is eventually consistent, and the team must reconcile provisional effects when the primary returns. The reward is a recovery path whose assumptions are visible: a separate platform, a bounded cache, explicit refusal modes, an independent switch, and tests that exercise the path before an emergency. Monzo's Stand-in is a concrete example of this design, but the pattern applies to any service where a short list of useful actions matters more than keeping the entire product online.

Originally published on Dispatch.

Serving Markdown to AI Agents Without Breaking HTTP Caches

Chen Yuan — Thu, 27 Aug 2026 10:19:12 +0000

The web already has the content

The web already has the content. The challenge is not that AI agents cannot fetch a URL. The challenge is that the representation they receive, typically the same HTML a browser renders, is optimized for visual display, not for machine consumption. HTML pages are laden with navigation, scripts, styles, and layout markup that add token cost and dilute semantic signal for retrieval or summarization. The interesting engineering problem is serving a cleaner, more token-efficient representation from the same URL while keeping existing caches intact. Content negotiation via the Accept header provides a standard mechanism, but it requires careful implementation to avoid fragmenting or poisoning caches.

What Accept: text/markdown changes

The Accept request header tells a server which media types the client can process. Browsers typically send a broad value like text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8. An AI agent that prefers Markdown can send Accept: text/markdown, text/html;q=0.9,*/*;q=0.8 or a simpler Accept: text/markdown. This is a signal, not a command. The server is free to respond with text/html if that is all it provides. The media type text/markdown is registered for Markdown documents and is the appropriate type to use for this representation.

What changes when a server supports this is the representation it returns. Instead of the full HTML page, the server responds with a Markdown document containing the core content. This is not a replacement for HTML; it is an alternative representation optimized for a specific use case. HTTP already has a mechanism for this kind of negotiation. The server decides which representation to serve based on the client's expressed preferences.

The HTTP contract clients and servers need

Content negotiation relies on a few precise rules. When a server supports both HTML and Markdown representations for a resource, it must evaluate the client's Accept header to choose the appropriate one. Parsing Accept is not a simple substring match. It requires parsing the media types and their associated quality values (q-values) to determine the client's preference order. A client sending text/markdown;q=0.8, text/html;q=1.0 prefers HTML, so the server should choose HTML when it has that representation. The */* wildcard can match any media type, but a specific media range takes precedence over it.

The response must then declare what it is serving via the Content-Type header. For Markdown, this is text/markdown; charset=utf-8. For HTML, it is text/html; charset=utf-8. The server must not return one type while declaring another. A Content-Type: text/html response to a request for text/markdown is a contract violation, not a fallback strategy. If no available representation matches the client's constraints and the server is unwilling to supply a default, it can return 406 Not Acceptable, as defined by RFC 9110. It should not silently label HTML as Markdown.

Caching is part of the feature

A Vary: Accept response header is not optional here. Without it, a cache, whether a browser cache, a reverse proxy, or a CDN, might store the Markdown response for a URL and serve it to a subsequent request that expects HTML, or vice versa. The Vary header tells the cache that the response varies based on the Accept request header. The cache can then use that request-header value when deciding whether a stored response matches, keeping HTML and Markdown variants from being mixed.

This is where the engineering gets precise. Vary: Accept enables correct caching, but it can also cause cache fragmentation. Clients can send many different Accept strings that describe roughly the same preference, leading to multiple cache entries for one URL and a lower hit rate. The cure is provider-specific cache-key policy or normalization, not an assumption that every CDN treats Vary the same way. A good rollout checks the CDN's documentation and inspects real cache behavior with more than one header spelling.

CDNs also have specific handling for the Vary header. Akamai's current documentation describes a policy in which arbitrary Vary values can prevent caching unless the property is configured for them. Other providers document different behavior, and those policies change. Treat Vary: Accept as an application contract and a CDN configuration item, not as a guarantee that every edge will cache the variants in the same way.

A small implementation that does not lie

The simplest implementation involves checking the Accept header in a middleware or request handler and serving the appropriate representation. Many frameworks have libraries for parsing Accept headers; do not write a custom parser unless you enjoy handling edge cases. The route fragment below uses Express and the accepts package. The content lookup is application-specific, but the negotiation and response headers are the part worth copying.

This example uses Node.js with Express and the accepts library:

const express = require('express');
const accepts = require('accepts');

const app = express();

app.get('/article/:slug', (req, res) => {
  const content = getContentForSlug(req.params.slug); // application-specific lookup
  const available = ['text/html'];
  if (content.markdown) available.push('text/markdown');

  // The package handles q-values and media-range specificity.
  const choice = accepts(req).types(available);
  res.vary('Accept');

  if (choice === false) {
    return res
      .status(406)
      .set('Content-Type', 'text/plain; charset=utf-8')
      .send('No acceptable representation is available.');
  }

  if (choice === 'text/markdown') {
    return res
      .set('Content-Type', 'text/markdown; charset=utf-8')
      .send(content.markdown);
  }

  return res
    .set('Content-Type', 'text/html; charset=utf-8')
    .send(content.html);
});

The available list contains only representations that the origin can actually produce. That detail prevents a request for Markdown from being selected when no Markdown body exists. It also makes an unsupported Accept choice visible as 406 instead of silently returning a mislabeled body.

For a static documentation site, the same idea can be implemented with explicit .html and .md files. The NGINX configuration below deliberately supports one narrow policy: it chooses the Markdown file only when text/markdown appears with no lower q-value. It is not a general Accept parser.

# http {} context. Each document has matching .html and .md files.
map $http_accept $doc_file {
    default $uri.html;
    "~*^text/markdown([[:space:]]*;[[:space:]]*q=1([.]0*)?)?([[:space:]]*,|$)" $uri.md;
}

server {
    root /srv/site;

    # /docs/guide -> /srv/site/docs/guide.html or /srv/site/docs/guide.md
    location /docs/ {
        types {
            text/html html;
            text/markdown md;
        }
        try_files $doc_file =404;
        add_header Vary Accept always;
    }
}

The example is useful when the representation is already stored as a file. If the endpoint must honor every q-value combination, keep negotiation in application code or a tested edge function instead of growing an untested regular expression. Either way, preserve Vary: Accept on successful responses and on errors generated by the variant route.

When a separate Markdown URL is safer

Content negotiation is elegant but not the only solution. Some sites may find it simpler or safer to publish their Markdown content at a separate, predictable URL. For example, example.com/article.md or example.com/markdown/article-slug. This approach avoids all the complexity of Accept header parsing and Vary header management.

It is also more predictable for agents. An agent can fetch the Markdown representation directly without relying on the server correctly implementing content negotiation. This is a defensible choice, especially for documentation sites or content repositories that already manage content in Markdown. A separate URL eliminates the risk of cache fragmentation because the URL itself is the differentiator. There is no need for Vary: Accept; the URL is the cache key. However, it also breaks the principle of one resource, one URL, and can lead to link rot if the Markdown URL is not treated as a canonical representation.

The decision is an engineering trade-off. Content negotiation is more elegant and maintains a single canonical URL. A separate .md URL is more robust and simpler to cache and test. For sites with existing, heavy CDN caching and complex request routing, the separate URL often proves safer to implement, as it sidesteps potential CDN quirks with Vary handling.

How to test an AI-ready endpoint

A testing checklist is essential for validating a content-negotiation endpoint. Run these requests against staging before production, and inspect both the status and the response headers.

For a default HTML request, omit Accept or use the broad value a browser sends:

curl -sI https://example.com/article | grep -E "Content-Type|Vary"

Expected: Content-Type: text/html; charset=utf-8 and Vary: Accept.

For an explicit Markdown request, ask for the representation directly:

curl -sI -H "Accept: text/markdown" https://example.com/article | grep -E "Content-Type|Vary"

Expected: Content-Type: text/markdown; charset=utf-8 and Vary: Accept.

Then test a q-value fallback. This client prefers HTML but can still process Markdown:

curl -sI -H "Accept: text/markdown;q=0.8, text/html;q=1.0" https://example.com/article | grep -E "Content-Type|Vary"

Expected: Content-Type: text/html; charset=utf-8 and Vary: Accept. The server should respect the higher q-value for HTML.

Finally, ask for a type the server does not provide and inspect the cache headers:

curl -sI -H "Accept: application/json" https://example.com/article
curl -sI -H "Accept: text/markdown" https://example.com/article | grep -E "Vary|CF-Cache-Status|X-Cache"

The first request should return 406 Not Acceptable when the server declines to supply a default. The second should show Vary: Accept plus whatever cache status headers the provider documents. A passing curl command does not prove that a CDN is configured correctly. Make repeated requests with different Accept strings and confirm that the edge serves the right variant without mixing them.

What adoption has to prove

Adopting text/markdown content negotiation is a practical step, but it solves only one problem: delivering a cleaner representation to clients that request it. It does not make a site "AI-ready" by itself. The Markdown representation must be of high quality. It must strip out the cruft of templates and scripts while preserving the essential content hierarchy. The conversion process from HTML to Markdown is itself a separate engineering task, and a poorly generated Markdown document can be as useless as the raw HTML.

The caching implications are real. A site that serves Vary: Accept will see additional cache entries. This is the cost of doing business. The benefit is a better experience for AI agents and, potentially, lower token consumption and faster processing for the client. The adoption has to prove that the benefit outweighs the cost. For many content-heavy sites, the cost is negligible when a CDN normalizes Accept headers. For others, the fragmentation could be a concern. The only way to know is to measure. Monitor cache hit ratios after enabling the feature. Watch for an increase in cache misses. If the hit ratio drops unacceptably, consider normalizing the Accept header at the CDN level or using a separate Markdown URL instead.

The web has always supported multiple representations of a resource. The Accept header is a mature part of HTTP, not a new agent-only protocol. If you want to try this pattern, start with one content route, generate Markdown from the same source as HTML, implement the negotiation with a tested library, set Vary: Accept, and observe the cache before expanding the rollout. If your CDN cannot give you a clear variant policy, use an explicit .md URL instead.

Originally published on Dispatch.

What Does an Agent Harness Actually Do? Building a Minimal One in Python

Chen Yuan — Mon, 24 Aug 2026 08:07:19 +0000

What Is a Harness, Really?

You have used a coding agent. You type a prompt, it edits files, runs tests, maybe even opens a browser. But when you look inside the request, you are not sending your prompt directly to the model. Between you and the LLM sits a piece of infrastructure that does almost all of the work. That is the harness.

An agent harness is the runtime scaffolding that turns a language model into an agent that can perform work. It drives model and tool calls, manages conversation state and context, applies approval policies, and keeps the agent progressing through multi-step tasks. In simpler terms: the harness is everything around the model. The model thinks; the harness acts.

Think of the harness as the operating system for your agent. It holds the system prompt, which sets the agent’s role, behavior, and decision-making style. It maintains the message history. It decides when to stop. It turns the model’s request for a tool into an actual function call in your runtime. Without a harness, you have a chat endpoint. With a harness, you have something that can actually do things.

The modern harness was not designed top-down from abstractions. It was born bottom-up out of coding agents solving real problems. Claude Code runs on one. DeepSeek Harness treats every capability—tools, skills, even the agent loop itself—as a swappable plugin, and it explicitly defines Agent = Model + Harness. The harness is the layer that makes the model useful in the real world.

The Four Jobs of a Harness

Every harness does four things. Nothing more, nothing less.

First, it holds the system prompt. That prompt is the agent’s job description. It tells the model who it is, what tools it has, and how it should behave. The user never sees it, but it shapes every response. Here is what one looks like:

system_prompt = """You are a coding assistant with access to these tools:
- read_file(path: str) -> str
- write_file(path: str, content: str) -> None
- run_shell(command: str) -> str

You have one job: help the user complete their task. When you need information,
call a tool. Do not guess. Do not make up file contents. When you have enough
information to answer, provide the final answer and stop."""

Second, it manages the message history. The harness keeps every user message, assistant response, and tool result in a single list. That list goes back to the model on every request. The harness does not just append; it trims, compresses, and summarizes when the context window gets tight.

Third, it translates tool calls into real execution. The model outputs a tool_calls array. The harness parses that array, invokes the corresponding functions in your environment, collects the results, and sends them back as role: "tool" messages. That round trip is the core of the harness.

Fourth, it enforces permissions. The model can ask to run any shell command or delete any file. The harness decides whether to allow it, ask for approval, or block it entirely. Claude Code runs every tool call through a permission check before execution. Your harness should too.

def check_permission(tool_name: str, arguments: dict) -> bool:
    if tool_name == "run_shell" and "rm -rf" in arguments.get("command", ""):
        return False  # block dangerous commands
    if tool_name == "write_file" and not arguments.get("path", "").startswith("./"):
        return False  # only allow writing to current directory
    return True

The Agentic Loop, Line by Line

The agentic loop is the reason–act–observe cycle that drives every step of an AI agent. The model generates a response, optionally invokes tools, observes the results, and loops until the task is done. That is it.

Here is a complete, runnable single-file Python agent loop using the OpenAI API. It handles tools, dispatch, and the role="tool" message round trip.

import json
from openai import OpenAI

client = OpenAI()

def get_weather(city: str) -> str:
    return f"The weather in {city} is sunny, 72°F."

def get_time(timezone: str) -> str:
    return f"The time in {timezone} is 3:00 PM."

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_time",
            "description": "Get current time for a timezone",
            "parameters": {"type": "object", "properties": {"timezone": {"type": "string"}}}
        }
    }
]

tool_map = {"get_weather": get_weather, "get_time": get_time}

messages = [{"role": "system", "content": "You are a helpful assistant with tools."}]
user_input = input("You: ")
messages.append({"role": "user", "content": user_input})

while True:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
        tool_choice="auto"
    )
    assistant_message = response.choices[0].message
    messages.append(assistant_message.model_dump())

    if assistant_message.tool_calls:
        for tool_call in assistant_message.tool_calls:
            tool_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            result = tool_map[tool_name](**arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result
            })
    else:
        print("Assistant:", assistant_message.content)
        break

That loop does exactly what every harness does. It sends the conversation to the model. It checks whether the model requested tools. If yes, it executes them and appends the results as role: "tool" messages. Then it loops. If no, it prints the answer and stops.

The tool_calls field on the assistant message contains the model's requests. Each call has an id, a function name, and arguments as a JSON string. The harness must send back a message with role: "tool", the same tool_call_id, and the result as content. That is the contract.

# When the model calls get_weather(city="Boston"), the assistant message looks like:
{
    "role": "assistant",
    "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {"name": "get_weather", "arguments": '{"city": "Boston"}'}
    }]
}
# Your harness runs get_weather("Boston") and sends back:
{
    "role": "tool",
    "tool_call_id": "call_abc123",
    "content": "The weather in Boston is sunny, 72°F."
}

Tools: Teaching the Model to Ask Instead of Guess

The model does not call functions. It outputs a structured request to call a function. The harness executes the function and returns the result. That distinction matters because it means the harness, not the model, controls what actually runs.

You define tools as a list of JSON schemas. Each tool has a name, a description, and a parameters schema. The description is critical. The model uses it to decide which tool to call and with what arguments. Write descriptions that are concrete and specific.

tools = [{
    "type": "function",
    "function": {
        "name": "get_user_by_email",
        "description": "Look up a user by their email address. Returns user ID, name, and role.",
        "parameters": {
            "type": "object",
            "properties": {
                "email": {"type": "string", "description": "The user's email address"}
            },
            "required": ["email"]
        }
    }
}]

The harness can also force a specific tool using tool_choice. Set it to {"type": "function", "function": {"name": "get_weather"}} to skip the model's reasoning and go straight to the tool. That is useful for routing or for tools that the model should always call first.

Real harnesses expose many tools. Claude Code keeps its toolbox deliberately small—eighteen tools in the snapshot captured in late 2025, every one sitting behind a permission gate. OpenHarness ships with forty-three tools covering file I/O, shell, and more. The harness validates the arguments against the schema before executing the function. If the model sends malformed JSON or missing required fields, the harness rejects the call and returns an error message to the model.

def validate_tool_call(tool_call, tool_schema):
    required = tool_schema["function"]["parameters"].get("required", [])
    args = json.loads(tool_call.function.arguments)
    for field in required:
        if field not in args:
            return False, f"Missing required field: {field}"
    return True, None

Context: The Window Is a Budget

Every request to the model has a context window. For GPT-4o, that is 128,000 tokens. That sounds like a lot until you start appending file contents, search results, and tool outputs. The harness must manage that budget.

The harness does not just append every message forever. It tracks token counts. When the conversation gets too long, it has to decide what to drop. The simplest strategy is a sliding window: keep the system prompt, the last N user-assistant exchanges, and the most recent tool results. Drop everything else.

def trim_history(messages, max_tokens=100000):
    # Keep system prompt (index 0). Then keep the most recent messages
    # until we hit the token limit. This is a simplified version.
    system_msg = messages[0]
    recent = messages[1:]
    # In practice, you would use tiktoken to count tokens.
    # For now, just keep the last 20 messages.
    trimmed = [system_msg] + recent[-20:]
    return trimmed

More sophisticated harnesses summarize old turns. They send a compressed version of the history back to the model. Some use a separate summarization model. Others just truncate. The choice depends on how much context your task requires.

The harness also manages the context across tool calls. When the model calls a tool, the harness appends the result and immediately sends the updated message list back to the model. That means the model sees the tool output in the next request, not in the same request. The harness is the intermediary that makes that round trip possible.

# After executing a tool, the harness appends the result and loops.
# The next model request includes the tool result in the message history.
messages.append({
    "role": "tool",
    "tool_call_id": tool_call.id,
    "content": result
})
# The loop continues. The model now sees the tool output.

Permissions: The Harness Holds the Leash

The model can ask to do anything. The harness decides what it is allowed to do. That is the permission system.

Every tool call flows through a permission check before execution. The check can be automatic (allow or deny based on rules), interactive (ask the user for approval), or hybrid (allow some operations, ask for others). Claude Code gates every tool call behind its permission modes. DeepSeek Harness routes approvals through a swappable plugin, so you can replace the whole approval flow the way you swap any other component.

The simplest permission system is a whitelist. Define which tools are allowed and which arguments are acceptable. Block everything else.

ALLOWED_TOOLS = {"read_file", "write_file", "search_code"}
BLOCKED_PATTERNS = ["rm -rf", "sudo", "chmod 777"]

def permission_check(tool_name: str, arguments: dict) -> tuple[bool, str]:
    if tool_name not in ALLOWED_TOOLS:
        return False, f"Tool {tool_name} is not allowed"
    if tool_name == "run_shell":
        command = arguments.get("command", "")
        for pattern in BLOCKED_PATTERNS:
            if pattern in command:
                return False, f"Command contains blocked pattern: {pattern}"
    return True, ""

The harness can also implement a confirmation flow. When the model requests a dangerous operation, the harness pauses, prompts the user, and only continues if the user approves. That is how coding agents avoid accidentally deleting your entire project.

def execute_with_approval(tool_name, arguments):
    if tool_name == "delete_file":
        print(f"Model wants to delete {arguments['path']}. Approve? (y/n)")
        if input().lower() != "y":
            return "Operation cancelled by user"
    return tool_map[tool_name](**arguments)

What We Skipped (and Why It Matters)

The minimal harness above works, but it is not production-ready. Real harnesses handle a lot more.

Streaming. Real harnesses start executing tools before the model finishes generating. That cuts latency. The model streams tool calls as it generates them; the harness kicks off the tool execution in parallel. Our loop waits for the full response before dispatching anything.

Parallel tool calls. The model can request multiple tools in one response. Our loop executes them sequentially. A real harness runs them concurrently and collects all results before sending the next request.

Retries and error handling. Tools fail. The harness catches exceptions, formats them as error messages, and sends them back to the model. The model can then retry with corrected arguments or try a different approach.

try:
    result = tool_map[tool_name](**arguments)
except Exception as e:
    result = f"Error: {str(e)}. Please try again with different arguments."

State persistence. The harness holds the conversation state in memory. If the process crashes, the state is lost. Real harnesses persist the message history, tool results, and progress to disk or a database.

Cost control. Every loop iteration costs money. The harness tracks token usage and can enforce a budget. If the agent exceeds the budget, the harness stops the loop and returns a partial result.

total_tokens = 0
MAX_TOKENS = 100000
while total_tokens < MAX_TOKENS:
    response = client.chat.completions.create(...)
    total_tokens += response.usage.total_tokens
    # process response...

Tool registration. Our loop uses a hardcoded tool_map. Real harnesses have a registry where tools are dynamically added, removed, and discovered. Plugins register their own tools.

Build One Yourself

You now know what a harness does. The next step is to build your own.

Start with the loop from Section 3. Add a system prompt that defines your agent's personality and available tools. Add a few tools that matter to you: read a file, search the web, run a shell command. Add a permission check that blocks dangerous operations. Run it with a real task and watch the loop go.

Then make it better. Add streaming. Add parallel tool calls. Add a token budget. Add persistence. Add a summarizer for long histories. Each addition teaches you something about how real harnesses work.

The harness is not magic. It is a while loop, a message list, and a dispatch table. The model does the thinking. The harness does the work. Build one, and you will never look at a coding agent the same way again.

Originally published on Dispatch.

Private AI Inference with Homomorphic Encryption: A Practical Guide to Computing on Encrypted Data

Chen Yuan — Sat, 15 Aug 2026 12:23:41 +0000

In 2009, Craig Gentry proved that it is possible to compute on encrypted data without ever decrypting it, and the result was widely treated as a theoretical curiosity. Sixteen years later, homomorphic encryption has crossed from conference papers into production pipelines: banks screen transactions against encrypted watchlists, hospitals run diagnostic models on data that never leaves their custody, and in August 2026 Google announced private AI features built on the same primitives. The gap between "possible in theory" and "usable in practice" is still wide, but it is no longer an argument against trying. This guide walks through what homomorphic encryption actually computes, how the CKKS scheme turns encrypted vectors into a workable substrate for machine learning, and the cost model that decides whether a private inference pipeline is worth building at all.

The Promise: Compute Without Reading

Ordinary encryption has a hard property: a ciphertext reveals nothing about the plaintext. AES-CTR, ChaCha20, RSA — all of them scramble data so thoroughly that an attacker holding the ciphertext and a supercomputer cannot recover the message without the key. That property is also the problem. If a server stores customer data encrypted at rest, every query requires shipping the data (or the key) somewhere a human or a process can read it. The moment the data is decrypted for computation, the confidentiality boundary moves from the storage layer to the memory of whatever process is doing the work.

Homomorphic encryption changes the terms. A homomorphic scheme is one where operations on ciphertexts correspond to operations on plaintexts: Enc(a) ⊕ Enc(b) = Enc(a + b). A server can add, multiply, and combine encrypted values and return the encrypted result, and the client — the only party holding the key — decrypts the final answer. The server learns nothing about the inputs, the intermediate values, or the output. For inference, this is the entire ballgame: the model owner never exposes weights, and the data owner never exposes the query.

Why Plain Encryption Breaks Computation

To see why this is hard, consider what AES does to a single byte. The S-box substitution and the ShiftRows/MixColumns rounds mix the input so completely that flipping one plaintext bit changes roughly half the output bits. That avalanche effect is what makes AES secure, and it is exactly what makes it unusable for computation. There is no way to run a + b on AES ciphertexts because the algebraic structure of the ciphertext has no relationship to the algebraic structure of the plaintext.

Fully homomorphic encryption takes the opposite approach. Instead of starting with a scrambling cipher and hoping arithmetic survives, it starts with an algebraic structure that naturally supports both operations. The classic construction works over polynomial rings: plaintexts are small polynomials, ciphertexts are pairs of larger polynomials, and addition and multiplication in the ring map to addition and multiplication of the encrypted messages. Security comes not from destroying structure but from noise — every operation grows a random error term that must stay small enough for decryption to still recover the message. Multiply too many times and the noise drowns the signal.

From Gentry's Bootstrapping to CKKS

Gentry's 2009 breakthrough had two parts. The first was the observation that a scheme with bounded noise can still be made unbounded: if the decryption circuit is shallow enough, the server can evaluate it homomorphically, producing a fresh ciphertext with reset noise. That process, bootstrapping, was the theoretical missing piece. The second part was a working scheme with a decryption circuit shallow enough to bootstrap. The catch was performance — early bootstrapping took minutes per operation.

The decade after Gentry produced a family of practical schemes. BGV and BFV handle encrypted integers with exact arithmetic. TFHE (CGGI) works over encrypted bits and is fast enough for small circuits like database lookups and comparisons. And CKKS, published by Cheon, Kim, Kim, and Song in 2017, introduced approximate arithmetic over encrypted real numbers, with a noise budget that behaves like floating-point error. That last property is what made machine learning viable: neural networks already tolerate small numerical errors, so a scheme that treats noise as precision loss fits the workload instead of fighting it.

CKKS also inherits a trick called SIMD packing from its predecessors. A single ciphertext in CKKS is not one number but a vector of hundreds of slots, and operations apply element-wise across the whole vector. A dot product — the inner loop of almost every inference step — becomes one ciphertext multiplication and a few rotations instead of hundreds of separate operations.

How CKKS Works Under the Hood

The construction is a polynomial ring R = Z[X] / (X^N + 1), typically with N = 4096 or 8192. Plaintexts are polynomials of degree less than N whose coefficients are the numbers you actually care about, scaled by a large factor and rounded. The scale factor is the scheme's version of fixed-point arithmetic: it sets how many bits of fractional precision survive a multiplication. After each multiply, the scale of the result grows, and the scheme provides a rescaling operation that brings it back down.

Noise is the real constraint. Each addition adds noise linearly; each multiplication roughly squares it. The scheme allocates a noise budget at encryption time, and every operation spends some of it. When the budget hits zero, decryption produces garbage. This is why parameter selection matters more in FHE than in any other part of a machine learning stack: the ring degree sets the maximum vector size and the top of the noise budget, the scale factor sets precision, and the multiplication depth you need sets how many levels the chain must provide before bootstrapping becomes necessary.

A minimal encrypted vector in Python, using a research-oriented binding, looks like this:

from openfhe import CKKSRNS, SecurityLevel, ScalingTechnique

# A fresh CKKS context with 40 levels of multiplicative depth
params = CKKSRNS()
params.SetMultiplicativeDepth(40)
params.SetScalingModSize(50)
params.SetSecurityLevel(SecurityLevel.HEStd_128_classic)

cc = CKKSRNS.GenCryptoContext(params)
cc.Enable(PKE)
cc.Enable(KEYSWITCH)
cc.Enable(LEVELEDSHE)
cc.Enable(ADVANCED)

keys = cc.KeyGen()
cc.EvalMultKeyGen(keys.secretKey)   # allow encrypted multiplication
cc.EvalRotateKeyGen(keys.secretKey, [1, 2, 4, 8])

plain = cc.MakePackedPlaintext([0.5, 1.5, 2.5, 3.5])
ct = cc.Encrypt(keys.publicKey, plain)

That is the entire setup. ct is now a ciphertext the server can store, transform, and return without ever seeing the values inside.

A Minimal Private Inference Pipeline

The classic private inference flow is split between a client that owns the data and a server that owns the model. The client encrypts its input, the server evaluates the model homomorphically, and the client decrypts the result. For a logistic regression — a single affine transform followed by a sigmoid — the encrypted computation is small enough to show in full:

def encrypted_predict(ct_x, ct_w, ct_b, cc, keys):
    # Encrypted dot product: one multiply, one add over packed slots
    ct_z = cc.EvalAdd(cc.EvalMult(ct_x, ct_w), ct_b)

    # Approximate the sigmoid with a low-degree polynomial
    # because division and exp are not directly supported
    sigmoid_approx = (
        0.5
        + 0.197 * ct_z
        - 0.004 * cc.EvalMult(ct_z, ct_z) * ct_z
    )
    return sigmoid_approx

Three details matter here. First, the sigmoid must be replaced by a polynomial approximation such as a Taylor or Chebyshev expansion, because CKKS supports only addition and multiplication. Second, the approximation degree is a direct trade against the noise budget: every extra multiply spends a level. Third, the whole vector of slots is processed in parallel, so one encrypted call classifies an entire batch of inputs, not a single sample.

For deeper networks, the same recipe repeats layer by layer: convolution becomes a sum of shifted and multiplied ciphertexts, ReLU becomes a polynomial like x^2-based approximation or a TFHE-style comparison, and pooling becomes rotations plus additions. The engineering problem is not expressing the model — it is keeping the depth inside the budget.

The Real Cost Model

Homomorphic inference is slow, and the honest framing is to say how slow and why. Ciphertexts are two or three orders of magnitude larger than the plaintexts they hold. Multiplication on a packed ciphertext is roughly a thousand to ten thousand times more expensive than the equivalent plaintext float operation, depending on ring degree and security level. Bootstrapping, when the noise budget runs out, costs on the order of tens of milliseconds to seconds per ciphertext — cheap enough to amortize over a packed batch, ruinous if applied per element.

The practical consequence is that private inference shifts the bottleneck from model quality to arithmetic budget. A model that runs in 2 milliseconds on plaintext floats can take seconds homomorphically, and the gap is dominated by the number of multiplications per slot, not the model's parameter count. Architectures that are friendly to FHE are the ones that keep multiplicative depth low: shallow MLPs, quantized networks, models with polynomial activations. Deep transformer stacks with attention and softmax are the worst case, because attention is a softmax followed by matrix products — softmax division is not a native operation, and its polynomial replacement is expensive.

When Private Inference Makes Sense

The honest answer is that homomorphic inference earns its cost in a narrow but real set of situations. The first is regulated data with a shared computation: hospitals collaborating on a model where patient records cannot leave each institution's boundary, or banks running joint fraud models over accounts they are legally barred from sharing. The second is API-based inference where the query itself is the secret — legal research, medical symptom triage, proprietary financial signals — and the client does not want the server to see what it is asking. The third is the emerging pattern behind the August 2026 announcements: consumer AI where the provider wants to process a user's data without being able to read it, as a product differentiator rather than a regulatory requirement.

What homomorphic encryption does not buy you is protection against a malicious server. A server that controls the evaluation can still drop requests, return garbage, or measure timing and access patterns. FHE guarantees confidentiality of the data against the server's curiosity, not integrity of the result against the server's malice. Teams that need both must add zero-knowledge proofs or commit-and-reveal protocols on top.

Choosing a Library

The ecosystem has consolidated around a few serious options. Microsoft SEAL is the reference implementation of BFV and CKKS in C++, with Python bindings via the pybind11-based extensions; it is battle-tested but expects the caller to manage parameters. OpenFHE is the community successor, actively maintained, and adds BGV, TFHE, and a unified API across schemes. TenSEAL, built on SEAL, provides a Pythonic API for CKKS over PyTorch tensors, though its maintenance has slowed. Zama's Concrete implements TFHE with a compiler that takes plain Python functions and emits bootstrapped circuits, which suits smaller integer workloads like lookups and decision trees. For a new project, the practical default is OpenFHE for custom CKKS work, and Concrete when the target workload is small and integer-shaped.

# OpenFHE exposes the same context pattern across schemes,
# so the pipeline above ports to BGV with two line changes:
params = BGVRNS()
params.SetMultiplicativeDepth(20)
cc = BGVRNS.GenCryptoContext(params)

The parameter selection itself is a black art that libraries are only beginning to automate. The rule of thumb is: choose the ring degree from the vector size and security level, choose the scaling modulus from the precision you need, then count the multiplicative depth of your exact model graph and add headroom for the approximations. Getting this wrong shows up not as a crash but as silent precision loss at the output — which makes an end-to-end test with known plaintexts the most important step in any FHE project.

A Decision Framework for Your First Private Inference Pilot

A useful heuristic before committing engineering time: if the plaintext model fits on a single machine and the deployment is a one-off computation, homomorphic encryption is probably the wrong tool — shipping the data under an agreement is simpler. If the computation is recurring, the inputs are sensitive, and the participants do not trust each other enough to share plaintexts, the calculus flips. Start with a single layer, measure the noise budget after every operation, and instrument the number of slots used per ciphertext. Most teams discover that the bottleneck is not the cryptographic primitives but the model's activation functions, and that replacing one ReLU with a polynomial buys more than a faster library ever could.

The field is also moving faster than its reputation suggests. Ciphertext compression, GPU kernels for CKKS multiplication, and programmable bootstrapping have each cut the effective cost by an order of magnitude within the last few years. The 2009 result was a proof that encrypted computation is possible; the current state of the art is a proof that it is affordable for the workloads where confidentiality is actually worth money. For anyone building AI products on other people's data, that is a gap worth watching — and a pilot worth running.

Originally published on Dispatch.

Why Your LLM Classifier Doesn't Need the Taxonomy: Hypothetical Classification with Embeddings

Chen Yuan — Fri, 14 Aug 2026 14:47:39 +0000

Classifying free-text queries into a fixed product taxonomy is one of the most common LLM workloads in production, and one of the most quietly expensive ones. A typical e-commerce catalog ships with 400 to 800 legal categories, each spelled out as a fully qualified path like Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables. Every time a search query needs a label, the whole vocabulary has to travel with the prompt: as a giant Pydantic Literal, as a JSON schema, or as a few hundred lines of enum text. Tokens are not free, and neither is latency. The prompt grows, the small model you wanted to use starts misbehaving, and the request that should have cost a fraction of a cent now needs the biggest model on the roster just to keep the output valid.

There is a cheaper pattern that most teams never consider. Instead of forcing the model to choose from the real taxonomy, you let a small, cheap model invent fake categories that sound plausible, and then resolve those hallucinations back into the real vocabulary with an embedding lookup. It sounds backwards. It works surprisingly well. This article walks through the idea, the implementation, the failure modes, and the measurements you should collect before trusting it.

The Standard Answer: Structured Outputs and the Taxonomy Tax

The conventional approach is structured outputs. You define the legal vocabulary as a type, hand it to the provider, and ask for a constrained decode:

from typing import Literal
from pydantic import BaseModel, Field

FullyQualifiedClassifications = Literal[
    "Furniture / Bedroom Furniture / Beds & Headboards / Beds",
    "Furniture / Living Room Furniture / Chairs & Seating / Accent Chairs",
    "Rugs / Area Rugs",
    # ... times 500
]

class QueryClassification(BaseModel):
    """Structured representation of a search query."""
    classifications: list[FullyQualifiedClassifications] = Field(
        description="Possible classifications for the product."
    )

Constrained decoding guarantees that the answer is drawn from the legal set, which is a strong property. But it carries a hidden cost that grows with the catalog. The schema has to be sent with every request, so every call pays the full vocabulary in input tokens. Classification is a high-volume, low-value-per-request workload, which is exactly the wrong place to spend tokens.

The second problem is that the guarantee is only as good as the model's willingness to stay inside the schema. Small models choke on a 500-element Literal; they either truncate it, or they start hallucinating categories that were never in the list, which defeats the entire purpose. The teams I have watched hit this wall usually respond by upgrading the model, and the unit economics of the whole pipeline follow them upward.

The Trick: Ask the Model to Make Things Up

The key realization is that the model does not need to know the taxonomy to know what kind of thing a query is talking about. A query like "brown coffee table" is obviously a piece of living room furniture. The hard part is not understanding the query; it is mapping that understanding onto a specific node in a tree with hundreds of leaves.

So stop sending the tree. Send an example of the shape of a classification, and ask the model to invent new ones:

hallucination_prompt = f"""
Your task is to create novel, never seen before furniture, home goods,
or hardware classifications that best fit a search query.

Product classifications look like:
Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables
Décor & Pillows / Decorative Pillows & Blankets / Throw Pillows
Furniture / Bedroom Furniture / Dressers & Chests
Kitchen & Tabletop / Kitchen Organization / Food Storage & Canisters
Baby & Kids / Toddler & Kids Bedroom Furniture / Kids Beds

Here is the query to generate classifications for:
brown coffee table
"""

The prompt now fits in a few hundred tokens instead of several thousand. The model has no legal vocabulary to violate, so it will happily produce a made-up path like Furniture / Living Room / Tables / Coffee. That output is useless as a label by itself. But it is a precise description of the query's meaning, written in the same style as the real taxonomy.

Resolving the Fake into the Real

Now the problem becomes: given a hallucinated path, find the real category it points at. Embeddings make this a nearest-neighbor search instead of a string-matching problem, which matters because the fake path and the real path share almost no literal characters.

The setup is small enough to live in memory. Embed every real category path once at startup with a compact sentence model, keep the vectors in a NumPy array or a tiny vector store, and embed the hallucinated path at request time. The label is the real category whose vector has the highest dot product with the fake one:

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
real_paths = [...]  # the full taxonomy, loaded once
real_vectors = model.encode(real_paths, normalize_embeddings=True)

def resolve(hallucinated_path: str) -> str:
    v = model.encode([hallucinated_path], normalize_embeddings=True)[0]
    idx = int(np.argmax(real_vectors @ v))
    return real_paths[idx]

For a catalog of a few hundred categories, this is a single matrix-vector product, well under a millisecond even on a laptop CPU. The expensive part of the pipeline is now the tiny LLM call, and that is the whole point: you have moved the cost from a big model with a giant schema to a small model with a short prompt plus a local lookup that costs nothing.

Why This Works: What Small Models Are Actually Good At

The pattern sounds like it should fail, so it is worth being precise about why it does not. Small models are bad at constrained generation over large vocabularies: their attention degrades as the constraint set grows, and their sampling drifts into invalid outputs. But they are good at open-ended paraphrase and at writing text in the style of an example. Inventing a plausible category path for "brown coffee table" is a style-matching task, not a constraint-satisfaction task. It is exactly the kind of fluent generation that even a 1B-parameter model does reliably.

The embedding model, meanwhile, is the component that actually holds the taxonomy. Sentence embeddings map both the fake path and the real paths into a space where meaning, not surface form, determines distance. Furniture / Living Room / Tables / Coffee and Furniture / Living Room Furniture / Coffee Tables & End Tables / Coffee Tables share almost no tokens, but they land close together because the embedding space was trained on paraphrases. Each component does the part it is good at, and neither needs to be strong at the other's job.

When Cheap Models Are Enough, and When They Are Not

The pattern is not a universal replacement for structured outputs; it is a trade. You should measure three failure classes before committing.

The first is vocabulary drift. A small model inventing categories will sometimes produce a path that is semantically generic, like Furniture / Tables, which resolves to whichever real node is closest in the embedding space. If your taxonomy has many tables, the nearest neighbor may be wrong even though the query was specific. This is the dominant failure mode, and its rate is a property of your taxonomy, not of the model.

The second is resolution ambiguity. Some queries are genuinely under-specified: "black stand" could be a phone stand, a monitor stand, or a plant stand. The hallucinated path will be confidently wrong because the model had no way to know. This is not a bug in the pattern; structured outputs fail the same way, but they fail by returning a confidently wrong label with a schema-valid appearance.

The third is taxonomy churn. If your catalog changes often, the embedding index must be rebuilt, and a newly added category is invisible until then. Rebuilding a few hundred embeddings takes seconds, so this is an operational detail rather than a blocker, but it needs to be part of the deploy pipeline.

The pattern shines when the vocabulary is large, the queries are short, and you want the classification step cheap enough to run on every request without thinking about the bill. It is a poor fit when the taxonomy is tiny, when labels must be exact strings with no tolerance, or when you cannot tolerate any rate of misclassification and prefer the deterministic failure of constrained decoding.

Guardrails: Handling Unresolvable Hallucinations

In practice you want a confidence floor on the resolution step, not just an argmax. The dot product between the hallucinated vector and its nearest real neighbor tells you how convinced the system is. A path that resolves at 0.45 cosine similarity is a guess; one that resolves at 0.85 is a lock. The cheap fix is a threshold with two fallback tiers:

def classify(query: str, threshold: float = 0.6) -> tuple[str, str]:
    fake = invent_categories(query)          # small LLM, short prompt
    real, score = resolve(fake)              # embedding lookup
    if score >= threshold:
        return real, "auto"
    return run_structured_classification(query), "fallback"

Below the threshold you can escalate to the structured-output path on a bigger model, or route to a human review queue. Because the cheap path handles the bulk of traffic, the escalation path only fires on the ambiguous fraction, and your average cost stays low even though the worst case still uses the expensive machinery. A second guardrail worth adding is a short denylist of generic leaves that are known to swallow unrelated queries, so a generic hallucination can be rejected before it resolves to a useless bucket.

Measuring Quality Without a Labeled Set

The objection to this pattern is usually "how do I know it is accurate?" You can get surprisingly far without a hand-labeled test set. Sample a few hundred real search queries from your logs, run both the cheap path and the structured-output path on the same queries, and diff the labels. Disagreement does not mean the cheap path is wrong, but every disagreement is a case to eyeball. In practice the disagreements cluster into two piles: cases where the cheap path is semantically right and the schema path was forced into a wrong but valid leaf, and cases where the cheap path resolved to a generic neighbor. The first pile is evidence the pattern is working; the second pile tunes your threshold.

Track the resolution score distribution as a metric. If the mean score drifts down over weeks, the model vendor changed something or your taxonomy drifted; either way you want to know before the misclassification rate moves. A small dashboard with three numbers, mean score, escalation rate, and disagreement rate against the structured path, is enough to run this pattern in production with confidence.

Putting It Together

The complete pipeline is short enough to hold in your head: a tiny model invents a category path from a prompt of a few hundred tokens, an embedding lookup resolves that path to a real leaf in under a millisecond, and a similarity threshold decides whether the answer ships or escalates. The taxonomy never travels with the request, the cheap model stays cheap, and the expensive machinery only sees the genuinely ambiguous cases.

Hypothetical classification is one of those ideas that looks like a hack until you measure it. The vocabulary is the expensive part of classification, and the embedding model lets you carry it once, locally, instead of shipping it on every request. If your classification workload is currently paying the taxonomy tax on a big model, it is worth a weekend experiment: replace the schema with a hallucination prompt, add the lookup, and look at the disagreement rate. The numbers will tell you quickly whether the backwards idea is the right one.

Originally published on Dispatch.