DEV Community: Sakura Sky

The CTO Playbook for Agentic Systems

Andrew Stevens — Fri, 18 Sep 2026 08:11:01 +0000

Today we are publishing The CTO Playbook for Agentic Systems on our white papers shelf. It runs to 71 pages across nine parts, and it was written by Andrew Stevens, our CTO and CISO (Stevens, 2026), over eleven months of conversations with engineering leaders running agents in production. It is published by Whitepaper Press rather than by us. We are hosting it here because its audience is the one we spend most of our time with.

Why now

Most of the engineering organisations we work with have stopped asking whether agents can write production code. The question has moved on to what happens to an organisation once a large share of its output arrives without a human having typed it, and the public telemetry on that has turned uncomfortable.

Faros Research's 2026 study, drawn from two years of data across 22,000 developers and more than 4,000 teams, reports throughput up on every measure it tracks, and alongside it a 242.7 percent rise in the ratio of production incidents to merged pull requests, bugs per developer up from 9 percent in its 2025 edition to 54 percent, and pull requests merged with no review at all, human or agentic, up 31.3 percent (Faros Research, 2025; Faros Research, 2026). Faros does not read that last figure as anyone deciding to skip oversight. It reads it as reviewers being unable to keep pace.

We would hold those findings a little loosely, since they are telemetry from organisations running one vendor's platform and DORA's 2025 survey reaches a more optimistic conclusion about whether strong engineering practice protects you (DORA, 2025). But if your own review load has started behaving anything like that, the shape of the problem is already familiar and the answers are not obvious.

What the paper works through

Nine parts, each closing with a specific deliverable and a set of questions to put in front of your own leadership team.

Part 2 treats verification as a staffed capability rather than an overhead, sizes it, and puts a formula behind the cost per 1,000 governed actions. It also works the case most write-ups avoid, which is what an agent programme looks like from the inside once it has stopped paying for itself, and when the right call is to shrink it.

Part 3 names seven engineering functions that need owners, with a level range and a compensation-band anchor for each, then raises the problem underneath the ladder: agents absorb a good deal of the work junior engineers used to build judgement on.

Parts 4 and 5 cover how to split work between a squad and its agents, and how to make decision rights auditable. Most first attempts collapse into a binary of what an agent may and may not do, which does not survive an incident review. The paper runs two axes instead, and is direct about the tension that creates between a containment target and a break-glass approval.

Part 6 rebuilds the delivery lifecycle around a different unit of deployment, and includes a sourcing table you can hand to procurement, with a buy-or-build call and an owner for every stage.

Part 7 is the leadership work: five conversations worth preparing for, and a first-90-days communication runbook where every announcement carries a precondition that has to be true before you make it.

Parts 8 and 9 charter a governance board, put fourteen metrics on one page for it, set out four layers of hard control, and sequence the whole thing across six phases with a gate before scale. The gate has five criteria and runs in both directions, which is the design decision we find most useful in it.

What you can actually use

Two things make this more than a read.

The nine deliverables. Every part closes with a named artefact rather than a recommendation, each with an owner, a cadence, and a description of what finished looks like. An agent inventory with named owners. A rehearsed and timed rollback runbook. A signed, dated declaration of which roadmap phase you are actually in and which gate criterion you currently fail. When we are asked to assess an agent programme, these are reliably the documents that turn out not to exist.

The readiness assessment. Twenty-eight questions in the appendix, scored 1 to 5, describing what a weak answer reveals rather than what a strong one sounds like. It is built to be scored in a board meeting. We expect to be asked to run it, and we would rather organisations ran it themselves first.

Who it is for

CTOs and VPs of Engineering with agents already shipping code, the directors and principal engineers designing the review and levelling systems underneath that, security architects who own the control architecture behind an autonomy tier, and the executives who hold engineering accountable.

Download the playbook. For the architecture it assumes underneath, The Trustworthy Agentic AI Blueprint is the deeper treatment and GATE is its implementable form. GATE is an open framework Andrew Stevens authors and maintains personally, not a Sakura Sky product.

If you would rather not work through the twenty-eight questions alone, that is what our Managed GRC service line is for, and a conversation is usually the quickest way to establish which of the nine artefacts you are missing.

Disclosure: The CTO Playbook for Agentic Systems is written by Andrew Stevens, CTO and CISO at Sakura Sky, and published by Whitepaper Press, with copyright held by the author. The white paper is a registration download and Sakura Sky receives the registration details. GATE is an open framework authored and maintained by Andrew Stevens personally under CC BY 4.0 and MIT, and is not a Sakura Sky product. The Trustworthy Agentic AI Blueprint linked here is likewise Sakura Sky and Stevens work, and the same weighting applies. Managed GRC Services is a Sakura Sky offering, and Sakura Sky provides advisory and managed services of the kind discussed here. Third-party research is cited as published and was checked on the dates given in the references.

Not legal advice. This article offers general commentary for an engineering leadership audience. The control-to-framework mapping described in the paper is directional and does not imply conformance with any regime or standard. Readers must obtain independent advice on how any of these apply to their circumstances.

References

DORA, 2025. State of AI-assisted Software Development. Google Cloud. Available at: https://dora.dev/research/2025/dora-report/ [Accessed 18 September 2026].

Faros Research, 2025. The AI Productivity Paradox Report 2025. 23 July. Faros AI. Available at: https://www.faros.ai/blog/ai-software-engineering [Accessed 18 September 2026].

Faros Research, 2026. Ten takeaways from the AI Engineering Report 2026: The Acceleration Whiplash. 12 April. Faros AI. Available at: https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways [Accessed 18 September 2026].

Stevens, A., 2026. The CTO Playbook for Agentic Systems, Version 1.2. Whitepaper Press. Available at: https://www.sakurasky.com/white-papers/cto-playbook-for-agentic-systems/ [Accessed 18 September 2026].

A 150B MoE on a Developer Laptop: An Upper Bound on What Faster Storage Buys You

Andrew Stevens — Wed, 16 Sep 2026 19:45:26 +0000

Frontier-scale models are being published as open weights faster than most teams have worked out what to do with them. The question I wanted answered was narrow and practical: can an engineer run one of these on the laptop they were issued, and if so, what is actually holding it back?

The use case is evaluation rather than production. Before anything gets deployed, somebody has to find out whether a model holds a house style across a long document, whether it calls tools without falling over, whether its output is worth the trouble at all. That work belongs on an engineer's own machine, using material that has not been sent anywhere, and it does not need throughput. It needs the model to run.

Conventional wisdom says it will not, or that it will crawl, and the reason given is always storage. A model this size cannot sit in memory, so the parts you are not using have to stream off a disk, and the disk becomes the wall. The engine used here documents a reference machine where one token takes 24 seconds and states plainly that the bytes have to arrive, so faster silicon will not help.

That was my hypothesis going in. On a machine with a current NVMe, storage would still dominate, and the number worth measuring was how badly.

Across a three minute generation the drive spent under eleven seconds actually reading anything. A disk with infinite bandwidth would have moved the result from 1.11 tokens per second to roughly 1.18.

That figure is a bound, not a benchmark. Your machine will produce different throughput than mine, but the ceiling on what faster storage could win is the part that transfers, and on any drive of this class it is small.

The mechanism

A sparsely-activated mixture-of-experts model has a large parameter count and a small activation count. A router picks a handful of experts per token per layer and the rest of the network sits idle for that token.

Colibrì, an Apache-2.0 inference engine by Vincenzo Fornaro, treats that property as a placement problem (Fornaro, 2026a). Rather than requiring the model to fit in memory, it stages VRAM, RAM and storage as one hierarchy: the dense part of the network stays resident, the routed experts live on disk, and the engine reads the ones the router asks for. It keeps a per-layer cache, records which experts a workload routes to, and pins the hot ones. The engine is one C file per model family with no BLAS and no Python at runtime.

The project is explicit that this is research rather than a product, and that its scope is wider than storage. Its stated goal covers "model formats, memory hierarchy, storage I/O, placement, scheduling, kernels, speculation, and CPU/GPU overlap", and it ships a fully-resident configuration in which the disk drops out of the decode path entirely. What follows measures one machine at one point on that hierarchy.

I ran it on my work laptop

An AMD Ryzen AI 9 365 with 20 logical CPUs and 61 GB of RAM, a single WD Blue SN5100 1 TB NVMe, an integrated Radeon 880M this engine cannot use, and Ubuntu 24.04.

Nothing about that is unusual for an engineering laptop in 2026, which is why I used it. Benchmarks published on eight H100s are interesting and I cannot act on them. I wanted to know what the machine on my desk would do, and I suspect most people reading this want the same. If you have a current NVMe, enough RAM to hold a dense set plus a cache, and 85 GB of free disk, everything below should reproduce. There is no accelerator anywhere in it, which conveniently removes the variable that usually decides these comparisons before they start.

Two things about the silicon are worth knowing before anyone reads too much into the CPU figures later. The part mixes performance and compact cores at different sustained clocks, and a laptop chassis holds a lower sustained power limit than a desktop would. Either could inflate CPU time per unit of work, and neither is something I measured.

The model was DeepSeek V4 Flash, in the REAP-pruned 150B variant published by puwaer: 150,128,549,111 parameters across 43 layers of 132 experts, routing to six per token, occupying 84.7 GB (puwaer, 2026). Routed experts are native fp4, which is why Colibrì reads the published checkpoint with no conversion step.

REAP is an expert-pruning method whose authors report near-lossless compression on code generation at fifty percent of experts removed, on models from 20B to 1T parameters (Lasby et al., 2025). I did not evaluate that claim. This post makes no quality argument at all, and the one answer reproduced below contains a factual error about its own architecture, which is worth holding on to.

Colibrì v1.11.0, built by its own setup.sh with gcc 13.3.0 at -O3 -march=native -fopenmp -flto. That build line matters more than it looks for a hand-rolled no-BLAS engine, since -march=native on Zen 5 is the difference between measuring the workload and measuring your compiler.

Measure the storage first

The engine refuses to plan without a storage measurement and marks it required. It ships its own benchmark, and the block size is not arbitrary: each expert's matrices are stored adjacently and read in one call, so the read that matters is several megabytes rather than several kilobytes.

M=~/models/dsv4-reap-150b/model-00003.safetensors
./iobench $M 13 200 1 1
./iobench $M 13 200 4 1
./iobench $M 13 200 16 1
./iobench $M 13 200 4 0

With O_DIRECT: 5.67 GB/s at one thread, 6.17 at four, 5.72 at sixteen. Buffered at four threads: 7.43 GB/s, though that figure is inflated by the page cache on a file written minutes earlier and should not be used as a baseline for anything.

One run per configuration, no variance reported, so the 8% spread across thread counts is not a result. What the numbers do support is that a single thread already reaches within 10% of the best figure observed, which is unsurprising given that a 13 MB application read is split into many device-level requests before it reaches the drive.

For scale, the reference machine in Colibrì's GLM-5.3-Flash documentation measures 72 MB/s at queue depth 1 and saturates near 200 MB/s, and that drive imposes a floor of 24 seconds per token on that model (Fornaro, 2026b). A 2026 consumer NVMe is roughly eighty times faster at the comparable setting. The floor that dominates that document is a property of one drive, which is worth knowing before anyone reads those figures as the cost of the technique.

The plan

RAM    39.6 GB budget · 8.8 GB dense · 5.9 GB runtime · 24.9 GB warm experts · cap 43/layer
disk   51.0 GB cold experts · 416.6 GB free
limit  disk expert misses
hit    33% projected expert residency

Two thirds of the expert set stays on disk and the planner names disk misses as the expected constraint. It disabled speculative decoding, on the reasoning that drafting widens the set of experts a token needs, and enabled read and compute overlap.

That overlap setting is load-bearing for everything that follows, and I will come back to it.

The self-tuning pass then swept eleven candidates: one OMP team size, three loader-thread counts, two smaller RAM caches, two CUDA paths that a CPU-only run still evaluates, and confirmation runs. None cleared its three percent improvement gate, so the defaults were kept. The profile recorded 0.92 tokens per second at a 66.75 percent hit rate.

The run

time ./coli run --model ~/models/dsv4-reap-150b --no-think --ngen 200 \
  "Write one paragraph explaining what a mixture-of-experts model is."

Two hundred tokens in 180.1 seconds: 1.11 tokens per second, or 0.90 seconds per token. Time to first token 17.2 seconds on a seventeen-token prompt. Expert hit rate 87.6 percent, from 46,665 hits against 6,597 misses across 53,262 selections.

The answer was coherent and said that mixture-of-experts models activate "typically just two or three" experts. This checkpoint routes to six.

What the disk actually cost

v4_tokens prompt=17 generated=200 expert_requests=53262 hits=46665 misses=6597 bytes=88197562368
v4_direct reads=4816 flock_reads=0 fallbacks=1781 payload_bytes=60599304192
timing time_to_first_token=17.175s after_first=162.946s
real 3m2.409s / user 46m32.525s / sys 0m22.993s

Two counters report bytes and they disagree, so it is worth saying which one this analysis uses and why. v4_direct reads plus fallbacks comes to 6,597, which is exactly the miss count, so payload_bytes at 60.6 GB is the physical I/O attributable to misses: one read per miss, averaging 9.19 MB. The larger 88.2 GB figure in v4_tokens does not divide into any plausible per-read size and is some other accounting. If you reproduce this, check both.

So: 60.6 GB delivered by a drive measured at 5.67 GB/s is 10.7 seconds of device time across a 180-second generation. Remove the disk entirely and the same work finishes in about 169 seconds, which is 1.18 tokens per second. The ceiling on infinitely fast storage is about six percent.

The calculation needs no claim about where the other 169 seconds went. It is bytes divided by measured bandwidth, with no timing in it.

Halving the cache

A bound is not a diagnosis. If misses are so cheap, the obvious test is to cause a lot more of them and see what happens. So I pinned the cache budget explicitly, dropped the page cache before each run so nothing was read from memory, and halved it.

sudo sh -c 'sync; sysctl -w vm.drop_caches=3'
time ./coli run --model ~/models/dsv4-reap-150b --no-think --ngen 200 --ram 20 \
  "Write one paragraph explaining what a mixture-of-experts model is."

	40 GB budget	20 GB budget
tokens/sec	1.045	0.885
expert hit rate	85.5%	66.2%
misses	7,719	17,998
bytes read	71.4 GB	167.5 GB
decode	191.3 s	226.0 s
user CPU	49m 10s	54m 39s
device time at 5.67 GB/s	12.6 s	29.5 s

Halving the cache cost 15% of throughput, so misses are not free and the engine's own advice to give it memory is right. My first draft implied memory had stopped paying at this configuration, and that was too strong.

The interesting part is the accounting. Decode got 34.7 seconds slower. Of that, 16.9 seconds is the extra device time from reading 96 GB more. The user CPU went up by 329 seconds, which spread across 17 threads is about 19 seconds of wall clock. The two together come to 36 seconds against an observed 35, which is close enough to say that a cache miss costs you roughly as much CPU as it costs you disk. Dequantising an fp4 expert, copying it into place, evicting something else and updating the bookkeeping is real work, and it scales with misses just as reads do.

That reframes what memory is buying. The README explains the benefit through read volume. On this machine about half the benefit arrives as CPU you do not have to spend.

And the drive still never becomes the constraint. Two and a third times the traffic raised decode time by 18%, and even at the worse setting device time was 29.5 seconds of a 226 second run.

Ruling out spin-wait

Before claiming any of that CPU time was arithmetic, there is an obvious objection. GNU OpenMP spins by default while waiting at a barrier, so a thread stuck behind a straggler or behind a loader sitting in pread burns CPU and computes nothing. High user time proves the threads were runnable and nothing more.

time OMP_WAIT_POLICY=passive ./coli run --model ~/models/dsv4-reap-150b --no-think --ngen 200 --ram 40 \
  "Write one paragraph explaining what a mixture-of-experts model is."

Decode 191.703 seconds against 191.290, user CPU 49m 16s against 49m 10s. Hit rate, bytes and read counts match to three significant figures. Nothing moved.

So the threads were not spinning at OpenMP barriers. One caveat I will not paper over: this rules out that specific mechanism, and the engine runs its own loader threads with their own synchronisation that OMP_WAIT_POLICY does not touch. Spinning somewhere else remains possible and I have no instruction counts to exclude it.

Those two runs also give something I was missing. 191.290 and 191.703 seconds at identical settings is 0.2% apart, so run-to-run variance on this machine is small enough to ignore at the resolution anything here is argued at.

Three things that cut the other way

Prefill was slower per token than decode. 17.2 seconds for 17 prompt tokens is 1.01 seconds each, against 0.90 in decode. Prefill is batched and should be faster per token. The likely explanation is a cold sweep through experts, which would make time to first token nearly pure I/O and roughly ten percent of the wall clock genuinely storage-bound.

One short prompt touched three quarters of the model. 4,216 distinct experts out of 5,676 in 200 tokens. Whatever the cache is learning, it is not learning that this workload uses a small corner of the network.

Residency and hit rate are different quantities. The planner projected 33 percent residency and the run recorded 87.6 percent hit rate. Those are a capacity ratio and a routing outcome, and the gap between them may be a skewed router rather than anything learned. The project lists learned pinning in its own open-hypotheses table, with the note that history "can overfit a prompt" and that the validation it wants is held-out cross-session A/B testing. Running one prompt repeatedly is the overfitting case, not evidence against it.

Where this leaves the project's advice

Colibrì's README says of this model family, in bold, "Give it RAM", and that the expert cache hit rate is what sets tokens per second, making --ram the single most valuable knob (Fornaro, 2026a).

The sweep says that is right, and says the stated mechanism is half the story. Memory does set throughput here. It sets it partly by avoiding reads, which is the documented reason, and about equally by avoiding the CPU cost of servicing a miss, which is not mentioned. If you are tuning a machine, that distinction matters, because it means the payoff from more memory does not disappear the moment your drive stops being busy.

There is a sharper version I got wrong in an earlier draft and want to correct in public. This model is 84.7 GB. A machine with 96 GB or more of usable memory holds all of it, at which point there is no streaming, no cache and no miss rate to improve, and the project publishes exactly that configuration. So the shape of the curve is: memory helps a lot while the hit rate is poor, helps less as it climbs, and then the moment the whole model fits, the streaming machinery and everything it costs in CPU goes away at once.

For comparison, the project's published ladder for GLM-5.2 at 744B runs from 5.8 to 6.8 tokens per second on six RTX 5090s with full residency, 1.8 on a 128 GB CPU-only desktop, 1.07 on a laptop-class box with an RTX 5070 Ti, and 0.05 to 0.1 on a 25 GB machine reading cold. Its DeepSeek-specific figure is about 1.6 tokens per second at 3k context on an RTX 5080 with two NVMe drives. The 128 GB desktop datapoint is worth sitting with, since it is 62 percent faster than this laptop and the obvious difference is memory.

What this does not say

One machine, one model, one prompt. Four runs, with repeatability of 0.2% between the two at identical settings, which is enough to trust the comparisons and nowhere near a benchmark suite.

It is also one engine, and that is the gap I would most want closed. The CPU time is real work, but it could be a property of this engine's hand-written kernels rather than of the workload itself. Running a comparable MoE through a second engine on the same box would separate those, and I have not done it. Until someone does, read the compute finding as being about this engine on this machine.

I also have no instruction counts, no device-level I/O trace, and no memory bandwidth figure, so where the CPU time actually goes is unmeasured. The share attributed to miss servicing comes from the difference between two runs, not from a profile.

And 1.11 tokens per second is slow. A 600-word answer is around ten minutes, extrapolated from a 200-token run rather than measured. That is workable for queued jobs and unusable for a conversation.

So can an engineer do this on their own machine

Yes, at 200 tokens in three minutes, which decides what it is good for.

Checking whether a model follows a style guide, holds context across a long document, or handles a tool-calling loop does not need throughput. It needs the model, a prompt and some patience. Queue the job, do something else, read the answer. Anything conversational is out.

The arithmetic that decides whether it will run at all is simple enough to do before installing anything. The dense set has to be resident, the expert cache wants whatever memory is left, and the free disk has to hold the whole checkpoint. This model needed 6.2 GB resident and 85 GB on disk, and 61 GB of system memory left 39.6 GB for the engine to divide. At 20 GB it still worked, 15% slower. On a 32 GB machine there would be very little cache left after the dense set, and the results above suggest that gets painful rather than impossible.

Storage is the part to worry about least. Any current NVMe is well past the point where the drive is what slows you down, and the two bounds are the argument: even after tripling the miss rate on purpose, device time was 13% of the run. Between a faster drive and more memory, the memory is what moves the number, and about half of what it buys is CPU not spent servicing misses.

The ceiling is worth naming too. Once the whole model fits in memory there is no streaming, no cache and no miss rate, and this checkpoint is 84.7 GB. Below that line you are trading against the cache. Above it the entire mechanism this post is about stops being relevant.

I should say what this does to our own position, since Sakura Sky runs managed environments for open-weight models and the measurement above cuts against part of the pitch.

Weights of this class run on an engineer's laptop. Anyone who has been told that large open models are out of reach without renting hardware has been told something that stopped being true. That makes local evaluation free, and it removes the excuse for not doing it before committing to anything.

What the laptop does not give you is fifty people using it at once, isolation between them, an audit trail, a recovery path, or an answer in under ten minutes. None of that is affected by anything measured here. What has changed is that nobody has to rent hardware to find out whether the model is any good first. Which for me, was the real win from this.

Reproducing it

The engine is at github.com/JustVugg/colibri, with its quick start, tuning notes and benchmark protocol in the same repository. The checkpoint is puwaer/DeepSeek-V4-Flash-0731-reap-150b on Hugging Face, and the engine notes for that family cover the flags used here. Neither the engine nor the checkpoint needed a conversion step, so an afternoon is a build, an 85 GB download and a run.

If you run it, record what the project asks for: hardware, commit, model container, exact command, prompt, cache state, throughput, time to first token, expert hit rate and bytes read. Add the build line and the page cache state, since a warm cache will hand you a number that means nothing. What I still cannot give you is an instruction count or a device-level I/O trace, and both would say more about where the CPU time goes than anything I measured.

The comparison worth making is not tokens per second between machines. It is the ratio of device time to wall clock on your own, because that tells you whether your next purchase should be a drive, more memory, or neither.

Disclosure: Sakura Sky designs and operates managed data, security and AI platforms for customers, including managed environments for open-weight models, which is a commercial interest in this subject and one the argument above partly cuts against. Colibrì is not a Sakura Sky project and we have no relationship with its author. The REAP-pruned checkpoint was published by a third party and its output quality was not evaluated here. All measurements come from a single machine on 16 September 2026 and are reproducible from the commands shown, which is not the same as being representative.

References

Fornaro, V. (2026a) colibrì: run frontier MoE models on hardware you already own. Available at: https://github.com/JustVugg/colibri (Accessed: 16 September 2026).

Fornaro, V. (2026b) GLM-5.3-Flash engine (c/glm53.c). Available at: https://github.com/JustVugg/colibri/blob/main/docs/glm53-flash.md (Accessed: 16 September 2026).

Lasby, M., Lazarevich, I., Sinnadurai, N., Lie, S., Ioannou, Y. and Thangarasa, V. (2025) 'REAP the Experts: Why Pruning Prevails for One-Shot MoE compression', arXiv:2510.13999. Available at: https://arxiv.org/abs/2510.13999 (Accessed: 16 September 2026).

puwaer (2026) DeepSeek-V4-Flash-0731-reap-150b. Available at: https://huggingface.co/puwaer/DeepSeek-V4-Flash-0731-reap-150b (Accessed: 16 September 2026).

Tollgate v0.2.3: What Long-Context Pricing Does to a Spend Cap

Andrew Stevens — Fri, 11 Sep 2026 09:21:15 +0000

Most teams find out what their long-context requests cost when the invoice arrives, and the gap is usually wider than the extra tokens explain. Some current models bill the whole request at a higher rate once the prompt crosses a size threshold, and a spend tool that stores one rate per token class has no way to represent that.

At the end of August I wrote about Tollgate, an open-source gateway that prices every LLM request by token, reserves the worst-case cost before forwarding, and refuses the request if that reservation would break a budget. Once the provider reports usage, the reservation is settled to the actual figure. That post described v0.1.1. Four releases later, v0.2.3 is out (Stevens, 2026), and most of the work in between went into the pricing rather than into the reserve and settle path itself.

Where a rate per token stops working

This is the part worth checking against your own provider's pricing page, because it is easy to miss and it costs money in one direction only.

Google prices Gemini 2.5 Pro at $1.25 per million input tokens for prompts up to 200,000 tokens and $2.50 above it, with output going from $10.00 to $15.00 at the same threshold. Gemini 3.1 Pro Preview uses the same 200,000 token threshold and the same 2x input and 1.5x output multiples at different absolute rates (Google, 2026a). The same tiering appears on Vertex AI, which is the Google surface Tollgate proxies (Google, 2026b). What re-rates is the whole request, rather than the excess above the threshold, so a prompt one token over the line costs twice as much on its input leg as the same prompt one token under it.

This is a property of individual models. Anthropic states that Claude 4.6 and later include the full one million token context window at standard pricing, and that a 900,000 token request is billed at the same per-token rate as a 9,000 token one (Anthropic, 2026). A provider that tiered last year may stop, and a new model family may arrive tiered.

For a proxy sitting in front of the model, that creates a specific problem. A price, as most systems store one, is one rate per token class. It cannot hold a rate that changes with request size. A proxy that stores "input costs $1.25 per million" and multiplies will under-charge every long request by the tier multiple, without any error to show for it. The ledger then disagrees with the invoice by an amount that looks like a rounding difference right up until somebody runs a long-context workload, at which point the same defect shows up as a several-thousand-dollar variance nobody can account for.

Migration 0010, in v0.2.2, puts a threshold and separate input and output multiples on the price row, resolved field by field against a deployment default. The tier applies to the reservation as well as to the settlement, so the two cannot drift apart on exactly the largest requests, which are the ones where drift is expensive.

Tiering stays off until you configure it, per model.

tollgate admin price set \
  --provider vertex --model gemini-2.5-pro \
  --long-context-threshold 200000 \
  --long-context-input-permille 2000 \
  --long-context-output-permille 1500

Defaulting to an uplift would invent a charge and break the meaning of the unit, since a million tokens would stop costing the per-million rate. Until the tier is set, a prompt above the threshold is logged as a possible under-charge rather than re-rated. I would rather leave a gap somebody can see than fill it with a number I made up.

What else changed since v0.1.1

It speaks the protocols the clients already speak. There is a native Anthropic route at POST /v1/messages, which takes the key as x-api-key and returns the upstream body unwrapped, and an OpenAI-compatible route at POST /v1/chat/completions that takes a bearer token. The Anthropic SDK, OpenAI clients, LiteLLM and Google ADK agents route through it by changing a base URL and a key, with no Tollgate SDK to adopt. Where a request does cross protocols, as an OpenAI-shaped request to a Google model does, the cost is metered from the usage the provider reports rather than from the shape of the request, because token accounting is usually the first thing a translation layer loses. Tollgate builds on proxy and cost-control patterns that LiteLLM established (BerriAI, 2026).

Streaming is metered from the stream. Usage is read from the provider's own end-of-stream reporting rather than estimated afterwards. A stream that does not finish cleanly is charged its full reservation rather than going unbilled, and the row is recorded as estimated, with /console/usage reporting measured_cost alongside estimated_cost. Keeping the measured figure separate from the assumed one has been more useful to me than the total, because it tells you how much of a month's spend number rests on an estimate.

Prompt-cache classes are priced per class, per model. A class left unpriced falls back to a conservative multiple of the input rate, 1x on reads and 2x on writes, and logs a warning naming the model. Several under-charges that had been present from the first release were fixed across v0.1.2 and v0.2.0: Gemini thinking tokens and tool-use prompt tokens were not billed at all, reasoning tokens through Vertex's OpenAI shim were not billed, and Anthropic cached requests were under-counted by up to 99%.

Budgets recover from losing their cache. Budget counters live in Valkey for speed and are rebuilt from the Postgres ledger. The reserve script used INCRBY, which recreates a missing key at the reserve amount, so a flush, a failover to a cold replica or an eviction restarted every budget's period with nothing logged, and left it restarted until somebody bounced the gateway. An absent counter carries no information about the spend that preceded it, and INCRBY treats it as a counter at zero. Reserve now refuses on an absent counter, rebuilds it from the ledger and retries once, and refuses the request outright if the ledger cannot be reached.

A refusal says whose fault it is. x-tollgate-reason now uses one vocabulary on every route. The native Anthropic path used to emit the SDK's error kind, so a budget refusal on /v1/messages reported permission_error while the same refusal on /v1/chat/completions reported budget_exceeded, and an alert keyed on the latter would have missed every Claude refusal. v0.2.3 extends that to failures. A pre-flight count that fails is now reported as an upstream error with a 502 rather than a backend error with a 503, which sends the operator to the provider instead of to Valkey and Postgres. A count endpoint that rejects a malformed body surfaces as a 400 instead of a 502, which SDKs had been retrying three times before anyone saw the real fault.

Privileged changes leave a trail. The audit_log table has existed with append-only triggers since migration 0002, and nothing wrote to it. Key issuance, revocation, budget changes and price changes now each append a row. The principal recorded is the OS user and host the command ran as, which is enough to line a change up against a shell history. It is not evidence that anyone was authenticated, and SECURITY.md says so.

If you are already running v0.1.x

Three of the fixes above change numbers you may have relied on, so they are worth acting on rather than reading past.

If you ran cache-heavy Anthropic traffic, or Gemini traffic using thinking or tools, on v0.1.1, the recorded cost for that period is low against the invoice, in the Anthropic cache case by as much as 99%. Reconcile that window against the bill rather than trusting the ledger for it.

If your Valkey instance was flushed, evicted or failed over to a cold replica while a budget period was open, that period restarted at zero and stayed restarted until the gateway was next bounced. Any cap that looked comfortable during such a window was not being applied as configured.

If you use exact admission on Vertex, it was refusing requests rather than counting them, because the body it sent carried fields the count endpoint rejects.

The upgrade path is tollgate admin migrate before starting the new binary, migration 0010 included, and a review of budgets before you start, because cache-heavy workloads will show higher spend once the classes are priced properly. Finish the rollout before relying on the counter repair. A v0.2.0 instance still recreates a missing counter at the reserve amount, so during a mixed rollout the old behaviour is live on the old instances.

Three bugs found by measuring the provider

Three of the fixes in v0.2.3 came out of a script that calls the providers' token-counting endpoints and prints what comes back. None were found by reading the code, and two contradicted a reading of the documentation that looked obvious at the time.

Vertex :countTokens rejects safetySettings, labels, toolConfig and cachedContent with a 400. An SDK-built body routinely carries safetySettings, so for most callers exact admission on Vertex refused the request instead of counting it, and the code that built the body looked reasonable. The endpoint accepts and counts contents, systemInstruction, tools and generationConfig, which is what is now sent.

generationConfig being counted matters more than it sounds. A responseSchema lives inside it and is billed as prompt material. A one-word message counted 1 token bare and 51 tokens with a small four-property schema attached. The schema had been dropped from the count on the reasoning that it describes the output rather than the input. That reasoning is tidy and it is wrong, and it under-reserved every structured-output request on Vertex.

The third was not a counting bug at all. reqwest has been built without its default features since the first release, to make the TLS backend an explicit choice, and that took http2 out with it. Every outbound provider request in every release up to now went over HTTP/1.1, with no connection multiplexing, on the busiest path in the product. A dependency's default feature list is not something most of us re-read after the first time we set it.

Where a provider's behaviour is the ground truth, measure it. Reasoning about which fields cannot possibly affect a token count is how two of those three bugs were written.

What it still does not do

The README carries a long limitations list on purpose, and it is the part I would read before the feature list.

The cap is a reservation, and a settlement can exceed it. Server-injected tool-use tokens, prompts that name a stored context cache, and long-context re-rating can all settle above what was reserved, and under the default fast admission the input estimate is deliberately rough. What the design aims for is direction rather than precision. It errs upward, because under-charging is the failure that makes a spend control worth nothing.

Provider-side context cache creation and storage are not visible from the request path. They bill against the cache resource, which no proxy in this position sees.

/metrics reports something as of v0.2.3, where it previously exposed a liveness gauge alone: request totals by decision, a cost total, and the configured limit for each budget. A scrape reads process memory only, and does not query the ledger, because an observability path that depends on Postgres and Valkey stops answering during the incident you would be using it to diagnose.

Streaming under exact admission is refused on the OpenAI route, and supported on the native Anthropic one. External media by reference is refused on both. The ledger drops partitions older than 90 days by default, which is a retention setting worth looking at before it becomes a gap in your own reporting.

It is beta and pre-1.0. Interfaces and the schema may still change.

Try it without infrastructure

cargo run -- demo

No Postgres, no Valkey, no cloud credentials. It prints an API key and two budgets, serves a mock provider and the console, and you can watch the per-key meter climb and hard-stop.

The open-source core remains single-operator and single-tenant, with keys and budgets managed through the CLI and an observe-only console. A Tollgate Enterprise edition will follow for teams running across several tenants and operators, adding web and API management, role-based access control and SSO, budget-change approval workflows, multi-org scoping, and audit export. If that edition is relevant to your environment, a GitHub Discussion or a note to Sakura Sky is the place to say so.

The repository is at github.com/sakura-sky/tollgate and the release is v0.2.3. If you put a real budget in front of real traffic and the ledger disagrees with your invoice, open an issue with the shape of the workload. That is the report I want most, because the failure this release was built around is one you only see on the bill. Please keep the SECURITY.md inbox for vulnerability reports.

Disclosure: I created Tollgate. It is published as a Sakura Sky open-source project under MIT, with copyright held by me and its contributors, and there is no contributor licence agreement. Sakura Sky intends to offer a commercial Enterprise edition alongside the free core, which is a commercial interest worth stating. Sakura Sky also builds and runs managed LLM gateway environments for customers, including LiteLLM deployments, and continues to do so; Tollgate is a narrower tool for a specific job and is not a migration recommendation for anyone. The provider prices quoted here were checked on the dates given in the references and change often.

References

Anthropic (2026) Pricing. Available at: https://platform.claude.com/docs/en/about-claude/pricing (Accessed: 11 September 2026).

BerriAI (2026) LiteLLM. Available at: https://github.com/BerriAI/litellm (Accessed: 11 September 2026).

Google (2026a) Gemini Developer API pricing. Available at: https://ai.google.dev/gemini-api/docs/pricing (Accessed: 11 September 2026).

Google (2026b) Vertex AI pricing for generative AI. Available at: https://cloud.google.com/vertex-ai/generative-ai/pricing (Accessed: 11 September 2026).

Stevens, A. (2026) Tollgate v0.2.3. Available at: https://github.com/sakura-sky/tollgate/releases/tag/v0.2.3 (Accessed: 11 September 2026).

Tollgate: Open-Source Hard Limits for LLM Spend

Andrew Stevens — Mon, 31 Aug 2026 12:39:10 +0000

In my experience most teams find out what their LLM workload costs from the invoice. Usage grows, a retry loop misbehaves, a feature ships with a larger context window, and the month closes on a number nobody forecast. Dashboards and budget alerts do not prevent that. They report money that has already left.

Tollgate turns the budget into an admission decision, taken before the request reaches the model. It is an open-source AI gateway and spend-control proxy, out now as v0.1.1 under MIT (Stevens, 2026).

What it does

Tollgate sits between your applications and their LLM providers, currently Anthropic and Google Vertex, both config-gated, plus a built-in mock for testing. There is no OpenAI adapter yet. Each request is authenticated, priced by token, and checked against every budget that applies to it before anything is forwarded. If a hard cap would be exceeded the request is refused and never reaches the provider. Spend is written to a durable, append-only ledger in Postgres, and a read-only web console shows live budget meters, the tokens-to-cost breakdown, and the gateway's own added latency.

It is written in Rust and deploys to Google Cloud Run. The Terraform module in the repository provisions the full stack and keeps the database URL and API-key pepper in Secret Manager, injected by reference rather than as plaintext environment values.

A refused request returns 402 Payment Required with an x-tollgate-reason header. An unroutable or unpriced model returns 400, also with a reason header.

That is worth unpacking, because the obvious alternative is a trap and the choice made carries its own caveat. 429 is the rate-limiting code, and provider SDKs commonly auto-retry it, so a monthly cap answering 429 would invite well-behaved clients to keep knocking at a gate that cannot open for weeks. 402 avoids that. But 402 is reserved rather than standardised, and the x402 payment standard, now a Linux Foundation project, is busy teaching agent clients that a 402 means pay and retry (x402 Foundation, 2026). Tollgate advertises no payment challenge and its callers authenticate with an issued key, so x402 middleware has nothing to act on here. If you run agents with payment middleware in the stack, check that interaction before relying on the status code alone. The reason header is there to be read.

What it does not do yet

The README carries a Limitations section. Three entries decide whether Tollgate fits a given workload today, so they belong here rather than three clicks away.

There is no streaming. Requests must be non-streaming, because usage cannot be metered from a partial stream in this release.

In the default admission mode, requests referencing external media by fileUri, file_id or URL are refused with a 400. The token cost of fetched media is not bounded by the size of the request body, so the reservation would be far too low to be safe. Those requests need exact admission, described below. If your traffic is multimodal, that decides your configuration on day one.

Prompt-caching cost is approximate. Cache-read and cache-write token classes are not yet priced separately, so on cache-heavy workloads the recorded cost can diverge from the provider's bill. Standard non-caching usage is exact. Metrics are minimal too: /metrics exposes only a liveness gauge, so Postgres and the console are where spend actually lives.

How a request moves through it

Pricing happens before the forward, which is what separates an enforceable cap from an advisory one. The worst-case cost of a request is reserved up front as a single atomic check-and-increment against Valkey, and once the provider responds with reported token usage the reservation is settled to the actual figure.

Four properties hold that together. Money is handled in integer micros with no floating point anywhere on the cost path. Postgres is the system of record and the Valkey counters are only a hot-path cache, reconciled from the ledger at startup in a direction that can raise a counter but never lower it. A request matching no budget at all is denied, so a missing configuration row cannot leave a deployment unenforced without anyone noticing. And a cache failure fails closed, which is the right call for a spend gate but carries a real cost: Tollgate is an in-path dependency, so if Valkey is unreachable your LLM traffic stops with a 503 rather than flowing unmetered.

Budgets resolve by scope: a specific key, a provider, a model, or the mandatory global backstop. Periods are daily, weekly, or monthly, and a request has to clear every scope that applies to it, so a per-key cap and a deployment-wide monthly ceiling operate together.

Two admission modes decide how strict the cap really is. Output tokens are bounded by the request's own max_tokens or maxOutputTokens. Input tokens are counted either by fast, the default, which estimates from the request body with no extra call and on token-dense input can settle a little over the cap, or by exact, which makes a pre-flight token-count call to the provider and enforces strictly. exact costs a round trip, and it adds a second provider dependency: if the count call fails the request fails with a 503 rather than falling back to the weaker estimate. If you need the cap strict to the token, that is the trade.

What v0.1.1 gives you

Per-key and global token budgets with a hard stop before the provider. Token-level cost accounting, currency-agnostic and integer-precise. Anthropic and Vertex adapters plus the mock. Budget and price changes that apply within the reload interval without a restart. Ledger retention by monthly partition, with partitions created ahead of need and old ones dropped. Per-request gateway latency on an x-tollgate-overhead-us header, with median and p95 in the console, so the overhead figure for your own traffic is something you read rather than something I assert.

To see it without provisioning anything, cargo run -- demo boots an in-memory gateway with the mock provider, prints a key and two budgets, and serves the console on loopback.

Tollgate builds on proxy and cost-control patterns that LiteLLM established (BerriAI, 2026), implemented in Rust.

Open core

The open-source core is single-operator and single-tenant by design. Keys and budgets are managed through the CLI, the console is observe-only with no write endpoints, and authorization on the observe endpoints is flat: any valid key can view the whole deployment's budgets and usage. Enforcement stays per key. A Tollgate Enterprise edition will follow for teams running across many tenants and operators, adding web and API management of keys and budgets, role-based access control and SSO, budget-change approval workflows, multi-org scoping, richer analytics, and audit export.

Try it

The repository is at github.com/sakura-sky/tollgate and the release is v0.1.1. Run the demo, and open an issue if it breaks in your environment. This is beta and pre-1.0, so interfaces and the schema may still change, and a report from someone who put a real budget in front of real traffic is worth more to me than any amount of design discussion. If the Enterprise edition is relevant to you, open a GitHub Discussion or contact Sakura Sky. Please keep the SECURITY.md inbox for vulnerability reports.

References

BerriAI (2026) LiteLLM. Available at: https://github.com/BerriAI/litellm (Accessed: 29 August 2026).

Stevens, A. (2026) Tollgate v0.1.1. Available at: https://github.com/sakura-sky/tollgate/releases/tag/v0.1.1 (Accessed: 29 August 2026).

x402 Foundation (2026) x402: an open standard for internet-native payments. The Linux Foundation. Available at: https://x402.org (Accessed: 29 August 2026).

GATE v1.5 Roadmap

Andrew Stevens — Mon, 31 Aug 2026 08:55:18 +0000

The v1.4 roadmap named six things: C20 output validation, break-glass formalisation, the OWASP AISVS and MITRE ATLAS mappings, gate-rust, gate-fuzz, and gate-knowledge. Five shipped in full, although C20 shipped with a streaming constraint I come back to below. gate-fuzz shipped one of its three deliverables, the Python-to-Rust differential harness. The bundle-derived strategy generator and the HTTP-level protocol fuzzer moved to v1.5, and the v1.4 release post named the deferral on the day. v1.5 delivers them.

v1.4 completed the boundary model: identity at instantiation, policy at execution, observation throughout, classification at delivery. v1.5 is about depth. Certification alignment, streaming, a reference implementation someone can actually run, and paying down the debt v1.4 named.

Here is what is coming.

ISO/IEC 42001, mapped with the asymmetry written down

GATE has claimed alignment with ISO/IEC 42001 since the first release, on the strength of a high-level theme table in the Standard Mappings appendix. A theme table is enough to start a conversation with an auditor and not enough to survive one.

v1.5 replaces it with a per-control mapping from C01 through C20 to 42001 clauses and Annex A controls, researched against the published standard (ISO/IEC, 2023) rather than against secondary summaries. It ships as gate-knowledge documents plus a machine-readable gate-conformance/mappings/iso-42001.yaml, in the same shape as the AISVS and ATLAS mappings but not with the same guarantee behind it. Those two validate against pinned open upstreams that the repository can carry a snapshot of. 42001 is a paywalled standard, so the yaml pins the edition and carries clause and Annex A identifiers, and the validator checks internal consistency rather than resolving each title against a shipped copy. That is a weaker check and the mapping header will say so.

The asymmetry document is the part that matters. 42001 is a management system standard: it asks for leadership commitment, a documented AI policy, competence and communication, supplier governance, and lifecycle planning that extends well past runtime. GATE addresses none of that, and it should not pretend to. The document lists each requirement 42001 makes that GATE does not touch, so a team reading the mapping can see the shape of the gap instead of inferring it. GATE conformance does not imply 42001 conformance, and after v1.5 the mapping will say exactly where the difference lies.

Sakura Sky plans a companion technical note alongside it, for CISOs reading the mapping in a 42001 certification context. It addresses the question the asymmetry document raises, which is what an auditor will still ask for once GATE is in place.

Streaming-aware C20

v1.4 ships output validation with a named constraint: streaming is disabled at the high_privilege tier for any agent whose action matrix can produce a hold or review obligation. You cannot classify a response you have not finished generating, and you cannot un-send a token. Rather than hide that, v1.4 wrote it into the spec as a limitation.

v1.5 closes it. Chunk-level classification with a sentence-boundary default, buffer-then-deliver semantics per chunk, per-tier classifier latency budgets so the buffering cost is bounded and stated, and a partial-classification event contract so the evidence trail covers a streamed response as well as it covers a single one.

Buffer-then-deliver is the design decision underneath all of that. A chunk is classified before it reaches the user, so there is nothing to take back. Token-level delivery with retroactive retraction stays out of scope. Once a user has read a token, withdrawing it does not undo the disclosure.

This one touches every repository. The contract in gate-contracts, the guardrail in gate-policies, a runner sub-check in gate-conformance, and the classification path in both gate-python and gate-rust.

gate-fuzz: converting PARTIAL to PASS

The runner reports 9 AUTOMATED and 11 PARTIAL by default, 11 and 9 when bundle stores are configured. Several of those PARTIAL results are not manual by nature; they are PARTIAL because closing them requires executing test scenarios a query tool cannot perform. gate-fuzz was scoped in June to close them, and v1.4 shipped the harness without the closure work. v1.5 ships the two deferred deliverables.

The first is Hypothesis strategies generated from the signed C09 invariant bundle, so invariants in the bundle produce property tests without anyone writing them by hand, for the invariant types the generator covers. That changes what a passing suite means: it becomes evidence about your bundle rather than evidence about the harness that tests it. The second is the HTTP-level protocol fuzzer, which sends mutation variants at a running Tool Gateway and treats any response that is not a schema rejection or a policy denial as a finding.

The framing that matters is the one the June roadmap set out and v1.4 did not deliver. With the README mapping each test to the conformance check it closes, running gate-fuzz alongside the runner converts PARTIAL results into verifiable PASS results, leaving in PARTIAL only the checks that need human judgement: controlled drills, process inspections, runbook sign-offs. v1.5 completes that scope, and the runner counts change accordingly.

The gate-rust conformance bridge

gate-rust can produce canonical JSON, envelopes, ledger events and signatures that match gate-python byte for byte, but it cannot hand the conformance runner evidence directly. A team running a Rust gateway still needs Python in the evidence path. That is its own commitment in v1.5, and one of the five known issues v1.4 named.

The bridge goes through either PyO3 or a subprocess boundary, so a single test suite runs over both implementations and produces byte-level proofs at the envelope, ledger, and signing layers. That is what unlocks the envelope-hash and ledger-event-hash byte-parity properties, which account for the four tests currently skipped in the gate-fuzz suite. A Rust gateway then produces runner-compatible evidence without a Python sidecar in the path.

gate-adk-reference: show me the code

The most common response to the framework paper is some version of "show me the code". The repositories answer it in pieces: a schema here, a policy bundle there, a library that implements the hashing. Nobody has published the thing those pieces add up to.

gate-adk-reference is an end-to-end GATE-governed data analyst agent built on Google's Agent Development Kit (Google, 2026), with sql-guard as the deterministic policy engine at the SQL tool boundary. Eight controls wired across all four layers. A BigQuery public dataset is the headline scenario, and a documented SQLite path covers air-gapped contexts where a cloud warehouse is not available.

The scope statement is part of the deliverable. This demonstrates a single integration pattern on one agent framework. It is not hardened, and copying it verbatim into production would be unsafe. That goes in the first screen of the README rather than the last.

A public playground

Reading about a policy denial is not the same as watching one happen. v1.5 plans a hosted, sandboxed instance of the reference agent where you can issue a prompt and watch the GATE event stream next to the chat: the policy decision, the quality gate on retrieval, the output classification, each appearing as the agent works.

Details are still being finalised, so I am not naming a hosting platform, a model, a domain, or a cost envelope here. Those land with the release.

AI Catalog and ARD interop

Scoped for v1.5, gated on the ARD specification stabilising. The final in-or-out call is not made.

When C17 discovers an unenrolled workload, GATE has no way to read an assurance claim the publisher has already made about it. Everything GATE knows about that workload, it has to establish itself.

Google and a set of industry partners published Agentic Resource Discovery in June, an open specification for publishing, discovering, and verifying AI capabilities across the web, covering MCP servers, A2A agents and OpenAPI tools, built on the Linux Foundation AI Catalog data model (Bu and Krishnan, 2026). Its trust manifest is the cryptographic layer, carrying identity and attestations for a published capability.

The work is small and bounded: an optional catalog_refs field on the ABOM and enrolment contracts, a verification helper in gate-policies, a signal recorded on the C17 enrolment decision when a reference verifies, and a conformance check on the integrity of catalog references where they are present. What a verified reference proves is that the publisher controls the domain the catalog is served from, not that the capability is safe to run, so it is recorded as provenance rather than treated as a substitute for the checks C17 already makes.

The boundary is explicit and I want it stated before the code exists. GATE consumes catalog entries. It does not publish them, and it does not extend the ai-catalog schema. Keeping the discovery layer and the governance layer separable means a change in one does not force a change in the other.

Contract debt, named and paid

Three items from the v1.4 known issues, each scoped for closure in v1.5.

The unified exception register: one exception_record contract covering the exception surface that C09 break-glass and the C17, C18, and C19 policies all consume, instead of adjacent shapes added one control at a time.

The multi-approver HITL Decision Record: approvals needing two or more approvers carried by the HITL record itself rather than borrowed from the break-glass path.

gate-python publication to PyPI, so the reference library installs the way a Python library is expected to install.

Together with streaming C20, the fuzz completion, and the conformance bridge, that accounts for every item the v1.4 release post listed under "What v1.5 picks up". None of them has been dropped without saying so.

The HTML edition becomes canonical

One meta change, and the gap it closes is a maintenance one. The cloud quickstarts and the day-2 runbooks currently live inside the paper, so correcting a quickstart means cutting a release. Page-number citations rot every time the paper is re-laid out.

From v1.5 the HTML edition at deterministicagents.ai is canonical and the PDF is an automated export of it. The quickstarts and runbooks move to live surfaces, cutting roughly thirty pages from the normative core. Citations point at section anchors rather than page numbers, which stop being stable between builds.

Timeline

No dates. Sequencing, which is the part I can actually commit to.

The framework paper update comes first, then the contracts, then the repositories that consume them, in dependency order. The ISO 42001 mapping is the substantive content workstream and runs alongside the paper rather than behind it. The reference implementation lands before the playground, and the playground goes last because it depends on everything above it.

Two open questions get settled at kickoff rather than in a later release. The AISVS candidate triage may produce new conformance checks, and if the triage confirms them they join the suite in v1.5. And the canonical-JSON float restriction documented in v1.4 is either the end state or it is not; the alternative is a shared float emitter across the Python and Rust implementations, and that call gets made before the contracts are cut.

If you are building agentic AI infrastructure and any of the above addresses a gap you have hit in practice, the issues and discussions at github.com/deterministic-agents are the right place to say so before the spec is locked.

GATE is published at deterministicagents.ai under CC BY 4.0 for the documentation and MIT for the code. The strategic companion to this framework is the Trustworthy Agentic AI Blueprint, co-authored with Sakura Sky.

Disclosure: GATE is authored and maintained by me personally rather than by Sakura Sky, and there is no paid tier, hosted version, or commercial product built on it. Agent governance is also my day job at Sakura Sky, which is a commercial interest worth stating. Two Sakura Sky items appear above: sql-guard, the policy engine named in the reference implementation, which is developed by Sakura Sky and released under Apache-2.0, and the planned companion technical note on 42001. Everything described in this post is planned work. None of it has shipped, and scope can change before it does.

References

Bu, J. and Krishnan, S. (2026) Announcing the Agentic Resource Discovery specification, Google Developers Blog, 17 June. Available at: https://developers.googleblog.com/announcing-the-agentic-resource-discovery-specification/ (Accessed: 19 August 2026).

Google (2026) Agent Development Kit (ADK). Available at: https://adk.dev/ (Accessed: 19 August 2026).

ISO/IEC (2023) ISO/IEC 42001:2023 Information technology - Artificial intelligence - Management system. International Organization for Standardization. Available at: https://www.iso.org/standard/42001 (Accessed: 19 August 2026).

Stevens, A. (2026) Governed Agent Trust Environment (GATE) v1.4. Available at: https://github.com/deterministic-agents/gate/releases/tag/v1.4 (Accessed: 19 August 2026).

GATE v1.4: Output Validation, a Rust Companion, and the Conceptual Layer as OKF

Andrew Stevens — Tue, 18 Aug 2026 19:16:56 +0000

GATE v1.4 ships today. It adds one control, three standard mappings, and three repositories, taking the Governed Agent Trust Environment from 19 controls to 20 across the same four layers: Identity and Integrity, Runtime Enforcement, Observability and Forensics, and Orchestration and Ecosystem. It is the largest release since I first published the framework in April. Existing C01 through C19 implementations remain compatible.

The release lives at github.com/deterministic-agents/gate/releases/tag/v1.4 (Stevens, 2026), with the artifacts bundle, SHA256SUMS, a single-file markdown export of the paper, and the 141-page PDF. The paper source is now public at gate-framework-paper. The framework home is deterministicagents.ai and the components sit under open licences at github.com/deterministic-agents. For the architectural rationale, "GATE: The Missing Infrastructure Layer for Agentic AI" remains the canonical introduction, and the v1.4 roadmap post from June sets out what I said this release would contain.

C20 Agent-to-Human Output Validation

GATE governed what an agent could execute at the tool boundary and what it could retrieve at the memory boundary. It said nothing about what the agent delivered at the end. C20 closes that.

The control sits in Layer 3, Observability and Forensics, alongside C13 intent telemetry and C19 drift monitoring. It performs per-response classification at the delivery boundary. Each final response carries a sensitivity tier, any regulated content categories it touches, a confidence score, and the obligations that follow: redact_fields, hitl_review, hold_for_review. At the high_privilege tier the control fails closed: a response whose classification matches no explicit pass-through entry in the signed action matrix is held, not delivered, enforced by the bundle schema and a policy guardrail rule. An output classification event is emitted per response, which gives the delivery boundary the same evidence coverage that tool envelopes give the tool boundary.

That is the fourth boundary in the model: identity at instantiation, policy at execution, observation throughout, classification at delivery. For most deployments the output gap is tolerable. For healthcare, legal advice, financial guidance, and HR decision support, obligations about what may be delivered, to whom, and under what review conditions are as real as obligations about what may be executed.

Break-glass becomes a contract

C09 has always required a signed record and dual approval for an invariant-halt override. What it lacked was a machine-verifiable format, so the record lived as a narrative document tied to a ledger event by convention.

v1.4 adds break_glass_record.schema.json to gate-contracts, with dual approval, scope binding, and expiry enforced by the schema. The invariant-halt ledger event now carries a break_glass_record_id referencing a valid record whenever an override was authorised, so a conformance runner can verify that each halt either resolves to one or was never overridden. That distinction previously required manual audit. This is a tightening rather than a new control. C09 stays C09 and the semantics do not change.

Two other controls tighten alongside it. C17 gains an automated enrolment fast-path backed by a signed policy contract, so discovered workloads can be enrolled without a human in the path when the policy says they qualify. C18 provenance now chains back to a registered source or an approved external feed through two normative URI schemes, which removes the ambiguity in what counted as an acceptable provenance claim.

OWASP AISVS, MITRE ATLAS, and NIST SSDF

The paper's standard mappings previously covered NIST AI RMF and ISO/IEC 42001, with EU AI Act alignment carried in the control text. v1.4 adds three more: OWASP AISVS, which reached 1.0 on 24 June and states testable security requirements across the AI system lifecycle (OWASP, 2026); MITRE ATLAS, the technique taxonomy for adversarial attacks on AI systems, whose tactics and techniques the mapping ties to the specific test scenarios C16 requires, so an adversarial validation harness has a threat model behind it rather than a list of categories (MITRE, 2026); and NIST SSDF, the secure development practice set issued under US Executive Order 14028 (Souppaya, Scarfone and Dodson, 2022), mapped as a deliberately narrow intersection: GATE touches SSDF at the artifact-integrity and tool-gateway surfaces (C03, C05) and marks everything else out of scope.

The mappings are pinned to upstream snapshots (AISVS v1.0, ATLAS content 2026.05) and machine-validated by a validator that ships in the repo: every referenced requirement and technique identifier must resolve against the pinned upstream with a matching title. Identifiers drift between revisions of a standard, and a mapping that has fallen out of date against its upstream is worse than no mapping, because it invites a reader to trust a claim nobody has rechecked.

The gaps are documented rather than papered over. Training data quality, model lifecycle management, and AI supply chain security sit outside GATE's scope, and the AISVS mapping says so. The June roadmap named these gaps as model evaluation, training data quality, and AI supply chain security; AISVS 1.0 organises the model-evaluation territory under model lifecycle management, and the mapping follows the published chapter names. The ATLAS mapping lists the techniques the framework does not address: the model interior is not a control-plane problem, and mapping to ATLAS does not make it one.

These mappings are informative. Passing GATE conformance does not imply conformance with AISVS, coverage of ATLAS, or certification against ISO/IEC 42001 or any other standard. They exist so an implementation team can trace a control to the requirement a reviewer will ask about. That is the whole purpose.

gate-rust v1.0.0

The Python library is the reference implementation and the right tool for prototyping gateways and building compliance tooling. It is the wrong tool for a high-throughput Tool Gateway written in Rust, where allocation patterns matter and Python interop adds latency.

gate-rust covers the primitives that have to be fast: canonical JSON, envelope construction, the hash-chained ledger, and ES256 signing and verification. The hash compatibility guarantee is the primary contract. gate-rust and gate-python must produce identical hashes for identical inputs, enforced by shared test vectors and a CI digest lockstep in both repositories. A single failing vector means the implementations disagree and both are wrong until one is fixed.

gate-rust covers the fast primitives only. Replay recording, schema validation, and the higher-level orchestration stay in Python where they make more sense. Neither library is on a package registry yet. v1.4 defers crates.io publication, so gate-rust installs from a tagged git ref (cargo add gate-rust --git https://github.com/deterministic-agents/gate-rust --tag v1.0.0), and gate-python publication to PyPI is a v1.5 item. Minimum supported Rust version is 1.78.

gate-fuzz v1.0.0

The v1.4 release ships one of three gate-fuzz deliverables originally scoped for the release. What lands is the Python-to-Rust differential harness that enforces byte equivalence between the two implementations at canonical-JSON, signing, and schema-validation layers. The other two deliverables - the bundle-derived Hypothesis strategy generator and the HTTP-level protocol fuzzer against a running Tool Gateway - move to v1.5. Naming the deferred items in the same release where the differential harness ships keeps the roadmap honest against what practitioners find when they clone the repo.

gate-knowledge v1.0.0

Google Cloud published Open Knowledge Format v0.1 in June, an open specification for representing curated knowledge as a directory of markdown files with YAML frontmatter, where concepts link to each other with ordinary markdown links and the only required field is type (McVeety and Hormati, 2026). It defines a file layout and nothing else. There is no SDK to install and nothing to run as a service.

GATE has the problem OKF was designed for. The control catalogue, the threat model, and the ABOM are all structured knowledge, and until now they lived as YAML, PDF, and HTML. An agent building a GATE-conformant system had to fetch several documents from several sources and assemble the picture itself.

gate-knowledge publishes the conceptual layer as an OKF v0.1 bundle: 20 control documents, the threat model, the adoption path, an ABOM template a team can fork and populate, typed relationship links between concepts, and a custom validator. The relationship links are the point. C17 links to C04 because discovery feeds the commission lifecycle. C09 links to C05 because invariants evaluate after policy. C19 links to C16 because they address different failure modes at the same tier and must not be merged. An agent navigating the bundle can understand a control's dependency surface before implementing it, without reading the full paper.

The bundle is the conceptual layer only. gate-contracts remains the normative schema source and gate-conformance remains the check source; nothing in gate-knowledge is authoritative over either.

Conformance runner v1.3.0

Check20 joins the suite for the new control. Check17 and Check18 move from PARTIAL to AUTOMATED when bundle stores are configured. A default configuration now returns 9 AUTOMATED and 11 PARTIAL results; a configured one returns 11 and 9. PARTIAL remains a statement about what the runner can see, not a failure: the report names the specific artefact each PARTIAL check still needs from the operator.

What is published in v1.4

gate-contracts v1.2.0 carries the normative JSON Schema definitions, extended with the C20 output classification event, the break-glass record, the auto-enrolment policy, and the approved feed registry.

gate-policies v1.2.0 carries the OPA/Rego policy and invariant bundles, adding C20 output classification with a fail-closed guardrail, C09 break-glass verification, the C17 auto-enrolment fast-path, and an explicit bundle manifest.

gate-python v1.2.0 remains the reference implementation, adding the C20 output module, break-glass verification, strict canonical-JSON hashing, and the cross-language test vectors gate-rust builds against.

gate-conformance v1.3.0 carries the twenty checks, the report template, the runbooks, the runner, and the three new machine-validated mappings.

gate-rust v1.0.0, gate-fuzz v1.0.0, and gate-knowledge v1.0.0 are the three new repositories, described above.

The framework paper source is public at gate-framework-paper, so the Quarto sources, control specifications, and diagram sources are now readable and forkable alongside the published PDF.

What v1.5 picks up

The June roadmap scoped gate-fuzz around PARTIAL closure: a README mapping each test to the conformance check it closes, and a suite that converts PARTIAL results into verifiable PASS results. That mapping arrives with the two deferred deliverables in v1.5, so the runner counts above stand until that mapping lands.

Alongside them, the paper's known issues section names five more: streaming-aware C20 classification, where a response is classified before it is fully formed; a unified exception register, meaning one exception_record contract for the exception surface that C09 break-glass and the C17, C18, and C19 policies all consume; a multi-approver HITL record, so approvals that need more than one approver are carried by the HITL record itself rather than borrowed from the break-glass path; gate-python publication to PyPI; and a gate-rust conformance bridge so the crate can produce runner-compatible evidence directly.

That is what v1.5 is scoped around, written down now for the same reason the June roadmap was. If you are building agentic AI infrastructure, deploying agents in a regulated environment, or advising organisations on AI governance, the issues and discussions on each repository are open. A bug report from someone who tried to implement a control is worth more to me than a comment on the paper.

References

McVeety, S. and Hormati, A. (2026) Introducing the Open Knowledge Format, Google Cloud Blog, 12 June. Available at: https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing (Accessed: 18 August 2026).

MITRE (2026) MITRE ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems. Available at: https://atlas.mitre.org/ (Accessed: 18 August 2026).

OWASP (2026) Artificial Intelligence Security Verification Standard (AISVS) 1.0, released 24 June. OWASP Foundation. Available at: https://owasp.org/www-project-artificial-intelligence-security-verification-standard-aisvs-docs/ (Accessed: 18 August 2026).

Souppaya, M., Scarfone, K. and Dodson, D. (2022) Secure Software Development Framework (SSDF) Version 1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities, NIST SP 800-218. National Institute of Standards and Technology. Available at: https://csrc.nist.gov/pubs/sp/800/218/final (Accessed: 18 August 2026).

Stevens, A. (2026) Governed Agent Trust Environment (GATE) v1.4. Available at: https://github.com/deterministic-agents/gate/releases/tag/v1.4 (Accessed: 18 August 2026).

Protecting the IP in a Generative World

Andrew Stevens — Tue, 18 Aug 2026 11:40:02 +0000

A studio found its own back catalogue inside the training data of a public model. Someone had run a handful of its older titles through a detection tool, and the tool reported, with reasonable confidence, that the model had seen them. The legal team moved quickly, because a decades-old library is the company's principal asset and defending it is the job. The engineering team was asked a simpler question and could not answer it. From the studio's own systems, could it prove when each of those works was created, what it was derived from, and whether anyone had ever been licensed to ingest it. The plain answer was no. The studio owned the content and could not produce the provenance.

That gap is the subject of this post. The studio's real exposure went well beyond a model having trained on its work. Nothing in its own architecture could establish the origin, ownership, and rights history of the things it makes, on demand and in a form an outside party would accept. The wider series argued that trust is something a system produces rather than asserts (see Trust Is an Engineering Output). For a media business, provenance is the specific shape that takes, and generative models are the forcing function that has finally made its absence expensive. This post works backwards from the discovery: what the old protections were built to do, what protection now demands, and the layer that closes the gap.

The back-catalogue discovery

Start with what the studio did have, because it was not nothing. Its digital rights management was intact. Streams were encrypted, playback was licensed, access to the finished assets was controlled, and outbound video carried watermarks. By the standard of the threat that DRM was built for, the studio was well defended. None of it touched the problem in the room.

What the studio lacked was a record, attached to each asset and maintained by its systems, of when the work was made, who contributed to it, what source material it drew on, and what rights sat over it. That information existed, scattered across contracts in a document store, production notes, and the memories of people who had moved on. It did not exist as data the studio could query. So when the question became "prove this is yours and prove what happened to it," the studio was reduced to reconstructing its own history by hand, which is exactly the position the earlier posts in this programme described as evidence that was never engineered.

This is the shape most media businesses are in, and it is worth being precise about why it is uncomfortable. Without that record, the studio could not prove ingestion had happened, could not cheaply assert ownership, and could not act on the opt-out mechanisms the law now provides, because it had no machine-readable way to express its rights across a library built over decades. The content was owned. The provenance was not producible. Those are different properties, and only the second one was being tested.

What DRM used to mean

Work backwards to how the protections were built, because they were built well for a threat that has changed. Digital rights management was designed to stop one thing: the unauthorised copying and redistribution of a finished asset. Encryption at rest and in transit, licence servers that decide who may play what, access controls, and watermarks that trace a leak back to its source. The protected object was the copy, and the moment of risk was consumption. For piracy, this was the right design, and for piracy it still largely works.

It was silent, though, on the things that now matter most. Encryption protects the delivery path, so it does block a crawler from ingesting that path, but the training-data problem mostly lives elsewhere: in the trailers, clips, and released titles that circulate in the clear, and in the studio's ability to express its rights and prove its own lineage. DRM protects the outbound stream. It says nothing about how a work may be used for training, nor about the origin story of the work behind the stream.

This is why DRM modernisation is not a matter of stronger encryption or tighter licence enforcement. The object that now needs protecting is not only the outbound stream. It is the provenance of the work itself, the verifiable account of what it is, when it was made, and what may lawfully be done with it. That account is data, and it has to be engineered as data.

What model-era IP protection requires

Once the threat moves from copying to ingestion, content IP protection needs three capabilities that traditional media rights management never had to provide, and the law on both sides of the Atlantic is moving toward them, even as the central US question, whether training counts as fair use, stays unresolved in the courts.

The first is machine-readable rights reservation. European copyright law gives rightsholders a text-and-data-mining opt-out, but it only functions if the reservation is expressed in a machine-readable form that a crawler can detect (European Parliament and Council, 2019). The AI Act then obliges providers of general-purpose models to put in place a policy to comply with that reservation and to publish a sufficiently detailed summary of the content used to train the model (European Parliament and Council, 2024). An opt-out a studio cannot express at scale, across a whole catalogue, is a right it cannot actually exercise.

The second is provable ownership. The United States arrives at a related point from the other direction: in a pre-publication report, the US Copyright Office takes the view that assembling a training dataset from copyrighted works implicates the reproduction right, because it involves downloading, storing, and copying those works (U.S. Copyright Office, 2025). Registration and a clean chain of title remain the legal instruments that let a studio sue and recover, so the provenance layer does not replace them. It feeds them, giving the studio a queryable evidence base for what it owned and when, which is the thing a claim or a defence eventually turns on.

The third is content provenance. The industry has converged on a standard for this, the C2PA content credentials specification, which attaches tamper-evident provenance and edit history to a media asset so its origin travels with it rather than living in a separate database (C2PA, 2026). Generative AI copyright disputes are often, underneath the legal language, arguments about who can prove what about a file, and content credentials are an attempt to make that provable at the level of the asset.

This is not hypothetical, and music is where it has been tested first. In July 2026 the Munich Regional Court found the AI music service Suno liable for copyright infringement in a case brought by the German collecting society GEMA, holding that specific protected works remained reproducible inside the models and that the outputs reproduced them, and that training carried out in the United States did not place it beyond German law (Music Business Worldwide, 2026). The decision is first-instance and under appeal, but it turns on exactly the question a provenance layer exists to answer: can a rightsholder show that a particular work was used, and that a particular output reproduced it. The studios watching that case are asking whether they could prove the same thing about their own catalogues, and most cannot.

The provenance layer

Move forward from the diagnosis and the resolution is an IP-aware data layer, engineered before the dispute rather than assembled during it.

It does three things, and they are the implementations of the capabilities above. First, it makes provenance a property of every asset: content credentials are attached at the point of creation, and the studio keeps an internal record of each work's creation date, contributors, source and derivation, and attached rights. That record is made tamper-evident through cryptographic signing and hash-linking, so a later alteration is detectable, which is what turns a database into evidence. Second, it expresses machine-readable rights reservation consistently across the whole library, so a model training opt-out is something the studio asserts at scale rather than in principle. Third, it adds monitoring that checks whether the studio's works surface in public datasets or model outputs. That monitoring is probabilistic, the same reasonable-confidence signal the studio started with, so it works as an early-warning system rather than as proof, and describing it that way keeps the claim accurate. Building that layer out of a media company's scattered production and rights data is IP architecture, and it is a large part of what Sakura's Data & AI practice does for content businesses.

This is where the argument connects back to the rest of the programme. A provenance layer is the media form of evidence as an engineered property: the studio can answer, for any asset, where it came from and what may be done with it, on demand and with the signed record attached. That is data provenance doing the same job in a studio that a hash-linked transaction chain does in a bank, and it turns content provenance from a compliance aspiration into a running feature of the platform.

What happens when this is not engineered

The cost of skipping this is not only the litigation the studio is now in. It is broader, and some of it is opportunity rather than risk.

Without the layer, a media company cannot readily prove ingestion happened, cannot enforce an opt-out it has no way to express, and cannot assert ownership without a manual reconstruction every time. Each of those is a live exposure as the use of copyrighted works for AI training data becomes contested. The part most businesses miss is on the other side of the ledger. A market for licensing catalogues to model builders is forming: in the same music-industry fight, Warner Music settled its US infringement suit against Suno in late 2025 and signed a licensing partnership, even as Universal and Sony kept litigating (Music Business Worldwide, 2026). Those deals still close on corporate ownership and contractual warranties rather than on asset-level provenance alone, but a content owner that can show clean provenance diligences faster, carries less risk, and negotiates from a stronger position. The same missing layer that leaves the company exposed also leaves value on the table in the market its own content is helping to create.

The evidence that lets a media business defend its rights or license its catalogue is engineering output produced before the dispute, not a legal artefact produced after it. Building the record that lets a studio prove what it owns, express how it may be used, and show when a boundary was crossed is a media security and evidence problem before it is a legal one, and it is the ground Sakura's Security practice works on with a media company's data and rights teams.

References

Coalition for Content Provenance and Authenticity (C2PA), 2026. C2PA Technical Specification, Version 2.4. Available at: https://spec.c2pa.org/ [Accessed 18 August 2026].

Music Business Worldwide, 2026. Suno infringed copyright in GEMA case, German court rules. Reporting the Munich Regional Court first-instance judgment in GEMA v Suno, Case 42 O 763/25, 31 July 2026 (under appeal). Available at: https://www.musicbusinessworldwide.com/suno-infringed-copyright-in-gema-case-german-court-rules/ [Accessed 18 August 2026].

European Parliament and Council, 2019. Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market and amending Directives 96/9/EC and 2001/29/EC. Official Journal of the European Union, L 130, 17 May, pp. 92-125. Available at: https://eur-lex.europa.eu/eli/dir/2019/790/oj [Accessed 18 August 2026].

European Parliament and Council, 2024. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, L 2024/1689, 12 July. Available at: https://eur-lex.europa.eu/eli/reg/2024/1689/oj [Accessed 18 August 2026].

U.S. Copyright Office, 2025. Copyright and Artificial Intelligence, Part 3: Generative AI Training (Pre-Publication Version). United States Copyright Office, Washington, DC, 9 May. Available at: https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-3-Generative-AI-Training-Report-Pre-Publication-Version.pdf [Accessed 18 August 2026].

Your System Prompt Is Not a Trust Boundary

Andrew Stevens — Tue, 18 Aug 2026 09:28:15 +0000

A text-to-SQL agent is one of the easiest AI features to demo and one of the more awkward ones to get through a security review. A non-technical user asks a question in English, a model writes SQL, the warehouse runs it, and a table comes back. The demo takes an afternoon. The review tends to stall on a harder question: what actually stops the agent returning data the person asking is not entitled to see?

In the reviews I sit in, the first answer is usually a sentence in the system prompt. Never return customer email addresses. Only query the orders table. Do not run anything expensive. Those sentences are worth writing, and OWASP's own first mitigation for prompt injection is to constrain model behaviour in exactly that way, with specific instructions about the model's role and limits (OWASP, 2025). My argument is not that the sentences are useless. It is that they are the wrong artefact to point at when somebody asks you to demonstrate that the rule holds, because an instruction sitting in the context window is subject to the same forces as every other token in that window.

Willison, who coined the term prompt injection by analogy with SQL injection, puts the mechanism plainly: LLMs follow instructions in content, and they cannot reliably distinguish the importance of instructions based on where those instructions came from (Willison, 2025). OWASP ranks prompt injection first in its Top 10 for LLM applications and is unusually direct about the outlook, noting that given the stochastic influence at the heart of the way models work, it is unclear whether any fool-proof method of prevention exists (OWASP, 2025).

In a text-to-SQL agent the injection surface is wider than the user's question. Tool results reach the context, as do column descriptions pulled from the information schema, and a returned row with instructions written into a free-text field. Any of those can carry the argument that talks the model past the rule you wrote.

My working assumption, and I would not claim it as more than an assumption, is that a rule an LLM enforces is a rule an LLM can be argued out of. If you need to show an auditor that a rule holds, the rule has to live somewhere you can test it and be proven wrong.

OWASP's own mitigation list says as much further down, recommending deterministic code to validate adherence to expected output, and handling privileged functions in code rather than handing them to the model (OWASP, 2025). That is the design this post is about.

What a second model buys you, and what it does not

The reflex fix, once a team accepts that the system prompt is porous, is to add a judge: a smaller LLM that looks at the generated SQL and rules on whether it is safe.

I think that makes a reasonable detective control and a poor floor. It costs tokens on every query and adds a network round trip to a path users experience as latency, which are the boring objections. The one I care about is that a judge is a probabilistic classifier, and the figure these systems tend to advertise sits somewhere around 95 percent. Willison's line on the guardrail vendor category is that in web application security 95 percent is a failing grade (Willison, 2025), and I would apply the same standard here. A control that is right most of the time can sit above your floor, and I would not build a floor out of one.

Where a judge does earn its place is on the classes a parser structurally cannot see, and I will get to those below: re-identification through innocuous columns, PII addressed by a string literal inside a JSON payload, aggregations that are each individually fine and jointly identifying. Those are semantic questions, and a grammar has no opinion on them. My preference is to run a judge above a deterministic layer where the budget allows, rather than in place of one.

Move the rule to where the SQL is

The SQL an agent produces is a string. Before it reaches the warehouse it is inert, fully inspectable, and, unlike natural language, has a grammar. That is the moment where a rule can be enforced by something that does not take arguments.

sql-guard is our attempt at that control. It is a policy engine that sits between the model's output and the warehouse client: it parses the SQL, runs an ordered list of rules against the syntax tree, and returns allow, confirm or deny. There is no LLM in the guard path. It is Apache-2.0, pure Python, and depends on sqlglot and nothing else. Install agent-sql-guard, import sql_guard, Python 3.11 or later (Sakura Sky Engineering, 2026).

from sql_guard import PiiDenylist, SqlGuard, SqlGuardConfig

guard = SqlGuard(SqlGuardConfig.from_settings(
    pii_denylist=PiiDenylist.from_mapping({
        "columns": ["email", "phone_number", "ssn"],
        "substrings": ["address"],
    }),
    allowed_tables=["my-project.analytics.orders"],
    dialect="bigquery",
))

decision = guard.evaluate_static(
    "SELECT customer_id, COUNT(*) FROM `my-project.analytics.orders` GROUP BY 1"
)
if decision.denied:
    return decision.reason

Six rules ship by default. Single SELECT only, so DML, DDL and multi-statement payloads are refused even when buried in a subquery. A PII column denylist. No SELECT * in any scope. Nothing whose columns the guard cannot enumerate. A table allowlist of fully-qualified names. And a cost cap with three thresholds: auto-execute below $0.10, ask for confirmation between, refuse above a $20.00 hard cap or a 10 GiB bytes-billed ceiling, all of them defaults you can move.

Parsing rather than pattern-matching is the load-bearing choice. Regex over SQL is famously brittle, and most of the bypasses described below would have been trivial against a regex while still being reachable against an AST walk that was not careful enough. sqlglot handles the dialect surface (Mao, 2026). BigQuery is the default and the one under heaviest real use. Snowflake, Postgres, Trino, DuckDB, ClickHouse and MySQL have test coverage, and Presto shares Trino's parser. The remainder of sqlglot's thirty-plus dialects should work without being battle-worn.

One caveat belongs here rather than in the limitations section at the end, because it decides whether any of this is a boundary at all. A library the caller has to remember to call is a convention. The host process's warehouse identity is the same whether the guard ran or not, so if another code path in the same process can reach the client directly, the guard is closer to a lint than a control. Putting the credential behind the guarded client, in a separate service account or a proxy the agent process cannot reach around, is what turns the convention into enforcement.

A parser will not be talked out of a rule. It can still be wrong about what the rule covers, and most of the rest of this post is eight examples of exactly that.

Eight ways the PII denylist was bypassed

v0.2.0 exists because an internal adversarial review of a deployed agent found two ways to get denylisted columns past the guard. Fixing those two surfaced six more. The release closes eight, each confirmed against 0.1.1 with a reproducing query before anyone touched a fix, each with a regression test that fails on the old code (Sakura Sky Engineering, 2026). The changelog itemises the work across ten entries, because two of the classes needed separate fixes.

Alias laundering. Rename a denied column inside a common table expression (CTE), the named subquery a WITH clause introduces, then project the alias from the outer query. Each CTE gets its own scope, and the rule called a helper that only read the outermost projection list, so the outer select named city and looked clean.

WITH c AS (SELECT billing_city AS city FROM `p.d.orders`)
SELECT city FROM c

Derived tables, UNION ALL arms and multi-hop alias chains through two CTEs all worked the same way.

Inner-scope stars. The star rule also ran against the outermost select only, so a SELECT * inside a CTE body or derived table went through untouched.

Stars inside a wrapping construct. A separate bug with the same effect. The check inspected the projection's root node, which meant OBJECT_CONSTRUCT(*) in Snowflake, COLUMNS(*) in DuckDB, * APPLY(f) in ClickHouse and ROW(c.*) in Trino all passed. It is now a deep walk, with COUNT(*) as the explicit carve-out.

Qualified t.*. In sqlglot a qualified star parses as a Column node wrapping a Star, not as a bare Star. An isinstance(projection, exp.Star) check walks straight past it, at the top level as well as anywhere else.

ClickHouse COLUMNS('regex'). Expands to an arbitrary set of columns and parses with no Star node anywhere in the tree to match on.

NATURAL JOIN. Joins on whichever columns the two tables happen to share. Without schema introspection the guard cannot rule out a denied column among them.

Column names that never become a Column node. sqlglot parses several positions as a bare Identifier, so a sweep collecting Column nodes missed them entirely: JOIN ... USING (email), column aliases of the AS g(email) form, and STRUCT('x' AS email) field names. The USING case was a working single-query value oracle.

Aggregates as a blanket exemption. Every aggregate function counted as PII-neutralising, so MAX(email), MIN(email), ARRAY_AGG(email), STRING_AGG(email) and ANY_VALUE(email) all returned real values. Only aggregates that reduce to a derived statistic qualify now.

Whole-row alias references. This one was not in the original report. It was found while fixing the others, and it was the most severe of the eight.

SELECT c FROM `p.d.orders` AS c

That parses as an ordinary column named c. The denylist had no denied name to match against, the star check found no star, and the guard auto-executed it. Strictly more powerful than the SELECT * it had been blocking since day one, and syntactically indistinguishable from selecting a column that happens to be called c. The fix also covers TO_JSON_STRING(c), ARRAY_AGG(c), STRUCT(c), the unaliased SELECT tbl FROM tbl form, CTE and derived-table names, and VALUES and PIVOT aliases.

Getting that rule to be usable took more care than getting it to be strict. Denying on a bare name collision would have broken WITH revenue AS (SELECT ..., SUM(x) AS revenue ...) SELECT revenue FROM revenue, which is a mainstream idiom, so the rule resolves the ambiguity from the AST instead: a table contributes only the name it is addressable by, and a CTE publishes its own output names.

The bypasses returned the success state

Most of the eight did not return deny. They returned confirm, and confirm is what static evaluation returns when no rule fires at all.

    for rule in self._rules:
        decision = rule.evaluate(ctx)
        if decision is not None:
            return decision

    return GuardDecision(
        outcome=GuardOutcome.CONFIRM,
        reason="Static checks passed; awaiting cost evaluation.",
        referenced_tables=tuple(sorted(tables)),
    )

So there was no near-miss and nothing to instrument. Eight queries reached denied columns, and the guard reported that static checks had passed, in the same words it uses for a query that is entirely fine. The whole-row alias went on through the cost gate and executed.

One thing worth knowing if you are building telemetry on this: GuardOutcome is shared across the two evaluation phases and carries a different meaning in each. In evaluate_static, confirm is the pass. In evaluate_cost it means ask the user before running this, and sits between the auto threshold and the hard cap.

A guardrail that fails closed tends to generate a support ticket and get fixed that afternoon. One that fails open often generates very little, which leaves going and looking as the main way anybody finds out. That asymmetry is my argument for treating adversarial review of this class of component as routine work, and it is why every fixed bypass now carries a test that fails against the version before it. A good share of the tests across those five files exist to keep the eight closed.

The fix broke our own demo

Tightening a guardrail means queries that used to work now do not, and 0.2.0 contains breaking behaviour changes on purpose.

The clearest casualty was the identity-resolution query bundled with the project as a worked example. It normalises email and mobile inside a CTE and projects only COUNTIF aggregates, an ordinary pattern in analytics that looks careful. It is now denied in both PII modes: the CTE scope projects the denied columns, and COUNTIF(email_norm = 'target') is itself a value oracle.

A WHERE clause is an oracle

sql-guard has two PII modes. "reference", the default, denies any mention of a denied column anywhere in the query. "project" denies only projections, checked across every scope.

Projection-only checking is the intuitive design and it does not hold, because a denied column in a predicate never appears in the output while still answering a question about itself.

SELECT COUNT(*) FROM `p.d.orders` WHERE billing_city = 'Columbus'

Run it again with LIKE 'a%', then > 'm', and you are doing binary search against a value you were never allowed to read. Depending on the cardinality, a handful of queries recovers it. GROUP BY, HAVING and ORDER BY leak the same way. The row count is the channel, and a rule that only inspects projections cannot see it.

Hence the strict default. If a deployment genuinely needs predicate access to a denied column, pii_mode="project" is there and the trade-off is stated in the docs. Narrowing the denylist, or pointing the agent at a pre-masked view the denylist does not cover, is usually the better move.

One nuance is easy to get backwards, and a careful reader of the changelog did: aggregation is not a safe harbour. COUNT, SUM, AVG and their relatives are treated as PII-neutralising only under pii_mode="project". Under the default mode no aggregate is exempt, because the whole point of the default is that the agent must not learn the values at all, and COUNT(*) ... WHERE email = ... learns them.

Limits of a parse-level SQL guard

These limits are structural. I would sooner publish them than have somebody discover them.

PII inside JSON, VARIANT or STRUCT payloads is not covered. JSON_VALUE(payload, '$.email') names only payload; the field name is a string literal the engine resolves at runtime. Denylist the containing column.

Two whole-row reads remain open by design. Selecting a STRUCT column whole, and an UNNEST alias over an array of structs, both return every field without naming one. Neither is distinguishable at parse time from the scalar-array form that is idiomatic and has to stay allowed. Same remedy: denylist the containing column.

Re-identification through non-PII columns is out of scope. If uid maps one-to-one to a person, blocking email does not stop correlation against an outside dataset.

Side channels remain. Row counts, dry-run byte figures and error messages all carry bits about denied values even when every direct reference is refused. The oracle above is the version we closed, and the family is larger than the fix.

There is no schema introspection. If you say a table is allowed, the guard takes your word for it. That constraint is the reason SELECT * is rejected everywhere instead of reasoned about.

The cost cap bounds a query, not a spend. evaluate_cost is per-call by construction. An agent in a retry loop issuing two thousand queries at nine cents each trips nothing. Cumulative exposure needs warehouse-side maximum-bytes-billed and a budget alert.

It is not an authorisation layer. Identity, IAM and row-level security sit outside it. It can approve a query that a correctly configured warehouse would have refused on identity grounds.

It is a boundary only where it is the only path to the credentials. Covered above, and in my experience the most common way this control gets downgraded to a suggestion.

The changelog also carries two known issues we have not fixed and one inconsistency: a top-level EXCEPT DISTINCT or INTERSECT is currently rejected as a non-SELECT, which fails closed and is an availability bug rather than a security one; allowlist breaches are under-reported in telemetry built on decision.reason because that rule runs last; and two spellings of hashed PII are handled inconsistently, in the direction of denial. Anyone evaluating this would do well to read that section alongside the README.

Defence in depth, and what the depth sits on

Warehouse-side column-level and row-level security are the durable answer to most of this. Policy tags on sensitive columns, masking policies, row access policies scoped to the calling principal: enforcement that lives with the data, applies to every client, and does not care whether the query came from an agent, a BI tool or somebody's notebook. Where a team can get there, I would push them to.

The gap sql-guard fills is between deciding that and having it. Warehouse-side controls need coordinated schema work, a data classification exercise that is usually half-finished, and sign-off from teams who own tables you do not. In the programmes I have watched, that runs to quarters rather than weeks. A denylist and an allowlist in a config file is an afternoon, it gives you a cost cap that column security does not, and it keeps working as a second layer once the warehouse work lands.

Message-boundary guardrails such as NeMo Guardrails or LangChain's belong in the same picture. They watch intent in the conversation, this watches the query at execution, and the failure modes look different enough to me that running both is usually worth the tokens.

Where it is

v0.2.0 is on PyPI as agent-sql-guard and the source, changelog and security policy are on GitHub. Two naming notes, because both have caught people. The import is sql_guard while the distribution is agent-sql-guard, and the unqualified name sql-guard on PyPI is an unrelated data-quality package by a different author. 0.2.0 is also the first release published to PyPI at all, because the name collision meant 0.1.x never shipped there.

The package classifiers say Development Status 4, Beta, which is the accurate description. The whole thing is roughly 1,400 lines across three modules, small enough to read end to end in an afternoon. For a component sitting on a security boundary I would treat that as a feature rather than an apology, and I would rather people read it than take this post's word for anything.

If you are running an agent that writes SQL against anything sensitive, an hour spent trying to beat your own guard, whatever form it takes, is likely to pay for itself. The eight above are a starting list. If you find something in sql-guard, please use the disclosure route in SECURITY.md instead of a public issue: security@sakurasky.com or a draft advisory, with the SQL, the config, and the decision you expected against the one you got. Everything that is not a bypass is very welcome in the issue tracker.

Disclosure: sql-guard is developed and maintained by Sakura Sky and released under Apache-2.0. Sakura Sky uses it in client-facing agent work. There is no paid tier, hosted version or commercial product built on it. The findings described here come from an internal adversarial review of a deployment rather than a third-party security audit.

References

OWASP (2025) LLM01:2025 Prompt Injection, OWASP Top 10 for LLM Applications. OWASP Gen AI Security Project. Available at: https://genai.owasp.org/llmrisk/llm01-prompt-injection/ (Accessed: 18 August 2026).

Sakura Sky Engineering (2026) sql-guard: deterministic policy engine for LLM-generated SQL, v0.2.0. Available at: https://github.com/sakura-sky/sql-guard (Accessed: 18 August 2026).

Mao, T. (2026) SQLGlot: no-dependency SQL parser, transpiler, optimizer and engine. Available at: https://sqlglot.com/sqlglot.html (Accessed: 18 August 2026).

Willison, S. (2025) The lethal trifecta for AI agents: private data, untrusted content, and external communication, 16 June. Available at: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ (Accessed: 18 August 2026).

Arkhe: An Ontology Language for AI Systems

Andrew Stevens — Thu, 23 Jul 2026 10:25:57 +0000

Two weeks ago I argued that the ontology layer is where the recurring agentic failures get retired, and that the artifact doing the retiring should live in your repository rather than a vendor's console (Stevens, 2026a). I closed that piece with a demand: insist on a neutral, versioned, reviewable form for your entities, relationships, and permitted actions. The obvious follow-up question: neutral form in what language, exactly?

So I spent my evenings since building one.

Arkhe v0.2.0 shipped today: a small, open-source ontology language for AI systems. Arkhe is my personal project, written on my own time, Apache-2.0 licensed, free forever, with no CLA. It is a Sakura Sky product in no sense at all. I am writing about it here because the argument started here.

One file, three registers

An Arkhe module is a YAML file in git. It declares the three registers from the previous post: what exists, how it connects, and what may be done, by whom, with what trace.

entities:
  FinancialModel:
    keys: [model_id]
    properties:
      status: {type: state, values: [draft, in_validation, validated, production, retired], initial: draft}
      last_validated: {type: date, optional: true}

links:
  owned_by:
    from: FinancialModel
    to: TradingDesk
    cardinality: many_to_one
    reverse: models

actions:
  grant_production_use:
    target: FinancialModel
    guard: target.status == "validated" && months_since(target.last_validated) <= 12
    authority: head_of_model_risk
    audit: mandatory
    effects:
      - target.status: production

A head of model risk can read that file and object to line eleven. That property, reviewability by the person who owns the business concept rather than the pipeline, drives most of the design. Guards are written in CEL, Google's Common Expression Language, because it is small, side-effect free, and already the policy expression language of half the cloud estate (Google, n.d.). The world is closed: entities and actions that are absent from the file are absent from the system, and no inference engine conjures new ones. The philosophical tradition behind the word ontology is acknowledged and then firmly set aside; Gruber's definition, an explicit specification of a conceptualisation, is the whole ambition (Gruber, 1993).

The other load-bearing decision: the spec is a contract, never a runtime. Arkhe compiles your module into a neutral intermediate form, one tool contract per action, and emitters generate what your systems consume from there. Nothing of Arkhe executes in your serving path. If the project vanished tomorrow, everything it generated for you would keep working, and the YAML would still be yours to compile with something else.

What compilation buys you

The compiler pipeline is deliberately dull. Validate the module (structural rules from a published JSON schema, then semantic rules: key references, link endpoints, guard names, effect types). Generate contracts. Emit.

Two emitters exist today:

The first produces a native Python library of guarded functions: call grant_production_use on a model that was validated fourteen months ago and you get a refusal that names the failing clause, months_since(target.last_validated) <= 12, rather than an exception or, worse, a success. An agent wired to those functions inherits the refusal semantics without any prompt engineering.
The second emitter, new in v0.2, produces an Open Knowledge Format bundle: one markdown concept document per entity, link, and action, generated from the module's annotations, for LLM consumption.

I think of the pair as complementary halves: the contracts declare what an agent may do, and the OKF bundle declares what it should know. Both are regenerated from the same file, so they cannot drift apart.

v0.2 also added synonyms as first-class annotations, so grounding can match the vocabulary people actually use ("desk" for TradingDesk, "sign-off" for grant_production_use) while the canonical names stay stable, and resolved types inline on effects, so downstream consumers stop re-deriving what a write will do.

The release I am most pleased with, though, is the one users will never invoke: a second implementation. The Rust port of the validator now passes the same frozen golden fixtures as the Python reference. Porting surfaced three behaviours the spec had never actually decided, all inherited silently from Python's YAML library, including the venerable Norway problem, where no parses as a boolean. All three are now pinned strictly in a public ADR. My conclusion from that exercise is worth its own post: a language with one implementation has a spec by accident.

Who it is for

I wrote persona guides for the three audiences I keep meeting in this work, and they map onto the registers:

GRC professionals get controls as structure. authority: head_of_model_risk and audit: mandatory are enforced properties with a trace, and a refusal that cites its failing clause is audit evidence in a way that a prompt instruction never will be. The reviewable YAML is the control library; the git history is the change record your auditor keeps asking for.
Data engineers get the semantic layer as code, sitting in the same repository as the schema it describes, validated in CI with finding codes and line numbers like any other artifact. If you have ever maintained business definitions in a wiki while the warehouse drifted underneath them, the appeal of arkhe validate failing your pipeline should be immediate.
Architects get protocol independence. The tool-contract IR sits at the centre; MCP, OpenAI function schemas, and Google ADK tool definitions are all planned as thin emitters from it, which means the protocol churn of the past two years stops being a rewrite trigger. Choosing Arkhe commits you to a file format, and deliberately to nothing else.

The use case I most want to see built, and intend to build as the flagship module, is an ontology of the AI estate itself: models, agents, tools, datasets, evals, guardrails, and incidents, with actions like promote-to-production guarded on eval results. The industry currently manages its most consequential systems with less structural rigour than it applies to a customer table. That seems worth fixing with the same medicine.

What it refuses to be

Arkhe is a language and a compiler. There is no workspace, no operational database, no query engine, no agent runtime. Palantir productised the three registers inside a platform a decade ago (Palantir Technologies, 2026), and several open projects are now circling the same territory with runtimes attached; the prior-art section of the README names them rather than pretending the idea arrived from nowhere. The bet Arkhe makes is narrower than any of them: the durable artifact is the reviewable file and its compiled contracts, and everything stateful belongs to systems you already run. If that bet is wrong, the cost of having tried it is one YAML file you can still parse with anything.

v0.2.0 is on PyPI (pip install arkhelang), crates.io (cargo install arkhe), and GitHub, with the spec sketch, nine ADRs, the golden fixtures, and the persona guides. It is deliberately small. If the previous post convinced you the neutral form should exist, this one is an invitation to argue with a concrete candidate, ideally in the form of an issue with a failing fixture attached.

Disclosure: Arkhe is the author's personal open-source project and is unaffiliated with Sakura Sky; Sakura Sky has no commercial interest in it. The author also maintains GATE, an open framework for governed agent runtimes, with which Arkhe aims to interoperate on equal terms with any other runtime.

References

Google (n.d.) Common Expression Language specification. Available at: https://github.com/google/cel-spec (Accessed: 23 July 2026).

Gruber, T.R. (1993) 'A translation approach to portable ontology specifications', Knowledge Acquisition, 5(2), pp. 199-220.

Palantir Technologies (2026) Ontology overview, Foundry documentation. Available at: https://www.palantir.com/docs/foundry/ontology/overview (Accessed: 23 July 2026).

Stevens, A. (2026a) Your ontology is the asset. Stop renting it back. Sakura Sky, 10 July. Available at: https://www.sakurasky.com/blog/own-your-ontology/ (Accessed: 23 July 2026).

Stevens, A. (2026b) Arkhe: an ontology language for AI systems. Available at: https://github.com/arkhelang/arkhelang (Accessed: 23 July 2026).

Identity Is the Bank Now

Andrew Stevens — Wed, 15 Jul 2026 09:56:08 +0000

A customer who has banked with the same institution for years is known to it in a number of different ways:

To the onboarding system that ran the identity checks, the customer is a passport, an address, and a date on which verification was completed.
To the fraud engine, the same customer is a pattern of devices, locations, and spending that it scores in real time.
To the anti-money-laundering system, a risk rating and a stack of alerts, most of them cleared long ago.
To the app and the contact centre, a name, a photograph, and a service history.

Four example systems, four versions of one person, and they do not agree with one another. When that customer moves house, the bank often hears about it three times, because three of the four systems have no idea the other two have already been updated. The customer notices the seams. So, in a different way, do the people who defraud banks for a living.

Every retail bank has some version of this customer, and the fragmentation is not the failing of one institution. Fraud, financial crime, KYC, and customer experience grew up as separate disciplines, with separate teams, separate budgets, and separate systems, and for a long time that separation held up fine. It holds up much less well now, because the four have collapsed into a single engineering question, and banking identity is the name for the answer.

Does the bank hold one reliable, current, shared understanding of who its customer is, and can every function that needs it read from and write to that understanding in close to real time? The wider series framed identity as the point where trust stops being a claim and becomes an engineering output (see Trust Is an Engineering Output).

Inside a bank, identity is turning into the thing the whole institution runs on.

The seamed customer

Look first at what the seams cost, because they are expensive on both sides at once. For the customer, they show up as friction. A repeat verification for a transaction the fraud engine did not recognise, a wait while a legitimate payment is held, a request to re-confirm details the bank already holds, an address updated in one channel while another channel keeps posting to the old one. None of this is catastrophic. All of it, accumulated across millions of customers, is the texture of a bank that feels harder to deal with than it should.

The other side is where it gets serious.

The same seams are an attack surface, and financial crime lives in the gaps between the four systems. A synthetic identity, assembled from real and fabricated fragments, can pass onboarding because the KYC system checks documents rather than coherence, and then behave well enough that the fraud engine, which never saw the onboarding anomaly, has no reason to look twice. An account takeover exploits the fact that the fraud engine sees a device change the AML system will never hear about, while the AML system sees a transaction pattern the fraud engine treats as somebody else's problem. No single system holds the whole picture of the customer, so no single system can see the whole picture of the attack. The criminal's real advantage is not sophistication. It is that the bank's view of the customer is split four ways and the criminal's view of that same customer is whole.

How identity got fragmented

The fragmentation was built one sensible decision at a time. The KYC system arrived to satisfy onboarding obligations, and those obligations are real: the global standard requires a bank to identify and verify the customer, identify beneficial owners, understand the relationship, and monitor it on an ongoing basis (FATF, 2012). The AML monitoring system was bought separately, often from a different vendor, and keyed on accounts and transactions rather than on people. The fraud engine came in for real-time scoring, keyed on sessions and devices. The customer relationship system was built for service, keyed on a contact record. Each was the right tool for its job, procured by the team that owned that job, on its own timeline.

What none of them shared was a canonical idea of the customer. Each held its own identifier for the same human being, and nothing tied those identifiers together into one durable customer identity. So the bank ended up able to answer four narrow questions well and the one broad question, who is this customer and what do we currently know about them, not at all. This is the state most KYC architecture is in: strong at the point check, silent on the whole.

The regulatory current now runs the other way on both sides of the Atlantic, which is worth noticing. In the United States, FinCEN's customer due diligence rule already requires banks to identify and verify the beneficial owners behind their legal-entity customers and to keep that understanding current, and a February 2026 order recast the refresh obligation on a risk basis rather than as a mechanical repeat at every new account (FinCEN, 2016; FinCEN, 2026). In the European Union, the 2024 anti-money-laundering reform moves the bloc onto a single rulebook and stands up a central authority to supervise it, with directly applicable rules on customer due diligence and beneficial ownership designed to make that data consistent across institutions rather than bespoke to each (European Parliament and Council, 2024). Different regimes, one direction of travel, toward shared and coherent identity. A bank whose own AML data cannot be reconciled across its four internal systems is starting that journey a long way back.

What a unified identity layer looks like

The fix is a unified identity layer, and it is worth being precise about what that means, because it is not another copy of the data in a warehouse. It is one canonical, current, authoritative record of each customer, built by resolving the fragments the four systems already hold into a single entity. Entity resolution, the work of deciding that this passport, that device pattern, and that contact record all refer to the same real person, is the hard technical core of it, and it is never perfectly clean, which is exactly why it has to be engineered rather than assumed.

The layer holds the durable answer to who the customer is, and links out to the signals each function produces: the KYC status and its evidence, the AML risk rating and its alerts, the live fraud signals, the service history. The point is not that one team now owns everything. The point is that any function can see the whole customer without having to own all of the customer. The fraud engine can factor in that onboarding flagged something odd. The AML system can see that the fraud engine just watched the customer's device and location change. Onboarding can stop asking for what the bank already holds. The identity layer becomes the shared, low-latency, authoritative view that every other system reads from and contributes to, and building that view out of messy source data is a serious piece of identity engineering rather than a reporting exercise.

The security architecture that supports it

Here is the part that a bank has to take seriously before it builds any of this, because getting it wrong turns an asset into a liability. A unified identity layer concentrates the most sensitive data in the entire institution into one place. That is precisely what makes it valuable, and precisely what makes it the highest-value target the bank owns. Unify identity without hardening it and the result is a better-organised honeypot.

So the unification and the protection are the same project, and banking security has to be designed into the layer from the first day rather than added once it works. That means access that is fine-grained and purpose-limited, so the fraud engine reads the fields it needs for fraud prevention and nothing more, and the contact centre sees a service view that does not expose the full financial-crime picture. It means strong identity for the workloads and the people reaching the layer, not just for the customers described in it. It means tokenising the most sensitive attributes, so that a breach of one consumer does not spill raw identity data. And it means a complete, tamper-evident record of every read and write, both because a supervisor will eventually ask who saw what, and because the bank itself needs to know. Get this right and the concentration of identity is a strength. Get it wrong and it is the worst single point of failure a bank could design. This seam between unifying data and defending it is the ground Sakura's Security practice works on with a bank's identity teams.

What changes once it exists

Come back to that customer, and follow what a working identity layer changes for them and for the bank at the same time. The address update lands once and every channel reflects it. The legitimate payment clears because the fraud engine can see the customer is exactly who the rest of the bank already knows them to be. Onboarding to a new product takes minutes because the bank reuses what it verified years ago instead of starting over. The friction that made the bank feel hard to deal with mostly disappears, and it disappears for the same reason the bank gets safer.

Because the criminal loses the seams. Synthetic identities are harder to sustain when onboarding, fraud, and financial-crime signals resolve to one view that has to stay coherent over time. Account takeover is harder when the device change the fraud engine sees and the transaction pattern the AML system sees are looking at the same customer record rather than two strangers. Fraud prevention and financial-crime detection stop being separate contests the bank fights with half the picture each.

And the bank gains the property the whole series has been circling. It can answer, for any customer, what it knows and how it knows it, on demand and with the evidence attached. That is trust as an output of the architecture rather than an assertion in a policy, and it turns out to run on a clean identity foundation. A bank that builds one is not a slow bank: Xapo Bank reached production on a tightly governed stack in weeks rather than quarters, because the foundations were engineered once and everything else inherited them.

The unified identity layer that all four functions read from and write to has to be built out of the fragmented systems a bank already runs, and constructing that resolved, current, authoritative view of the customer is the work Sakura's Data & AI practice does inside financial institutions.

References

European Parliament and Council, 2024. Regulation (EU) 2024/1624 of the European Parliament and of the Council of 31 May 2024 on the prevention of the use of the financial system for the purposes of money laundering or terrorist financing (Anti-Money Laundering Regulation). Official Journal of the European Union, L, 19 June. Available at: https://eur-lex.europa.eu/eli/reg/2024/1624/oj [Accessed 10 July 2026].

Financial Action Task Force, 2012. International Standards on Combating Money Laundering and the Financing of Terrorism and Proliferation: The FATF Recommendations (updated). Financial Action Task Force, Paris. Available at: https://www.fatf-gafi.org/en/publications/Fatfrecommendations/Fatf-recommendations.html [Accessed 10 July 2026].

Financial Crimes Enforcement Network (FinCEN), 2016. Customer Due Diligence Requirements for Financial Institutions, Final Rule. 31 CFR Parts 1010, 1020, 1023, 1024 and 1026. US Department of the Treasury. Available at: https://www.federalregister.gov/documents/2016/05/11/2016-10567/customer-due-diligence-requirements-for-financial-institutions [Accessed 10 July 2026].

Financial Crimes Enforcement Network (FinCEN), 2026. Order Granting Exceptive Relief from the Beneficial Ownership Requirements for Legal Entity Customers. US Department of the Treasury, 13 February. Available at: https://www.fincen.gov/system/files/2026-02/FinCEN-Order-CCDExceptiveRelief.pdf [Accessed 10 July 2026].

Migrating a Live Real-Time Communications Platform from AWS to Google Cloud

Andrew Stevens — Tue, 14 Jul 2026 11:15:08 +0000

11Sight operates a real-time voice and video engagement platform, the communications backbone behind its AI agents for automotive and hospitality businesses. Calls are the product. An infrastructure migration that takes the platform offline for a weekend was never an option. Sakura Sky's Cloud practice moved the platform from AWS to Google Cloud with a phased hybrid strategy that kept it serving live traffic throughout, and held user-facing downtime at the final cutover to under five minutes.

The Challenge

11Sight's production environment had grown up on EC2 and RDS: a monolithic web application on VMs, a Jitsi-based conferencing stack with video bridges and recorders, and a PostgreSQL database holding the customer data that every call depends on. Three constraints shaped the engagement:

Live traffic, all the time. A real-time communications platform has no quiet maintenance window long enough for a big-bang cutover, so the migration had to run while customers kept making calls.
Prove everything before production. Cutover timing, VPN latency, and application behaviour on GKE all had to be validated against production-grade infrastructure before any customer traffic depended on them, which made a full rehearsal environment a first-class deliverable of the migration plan.
Modernization over relocation. The goal was to land on a cloud-native footing, with Kubernetes where it earned its keep and managed services for state, rather than reproduce the VM-centric estate on new hardware.

The Engineering

We designed a compute-first, three-phase hybrid migration that decoupled the application move from the database move, so each could be validated independently. Four engineering decisions carried the project:

Landing zone before workloads. The first deliverable was a Google Cloud foundation built from Sakura Sky's Enclave Terraform blueprint: organization structure, IAM groups, centralized logging and monitoring, a Shared VPC, and a Cloud HA VPN linking the AWS and GCP networks. Every subsequent resource was defined in Terraform, in repositories created inside 11Sight's environment from day one.
Rehearse the whole migration in staging first. We provisioned a complete staging environment (GKE for the containerized web application, Cloud SQL for PostgreSQL 16, Memorystore with the Valkey engine, Compute Engine instances behind autoscaling groups for the conferencing workloads) and used it to rehearse the entire migration. That rehearsal validated that VPN latency between the GCP application tier and the AWS database was within production thresholds, and produced a measured downtime estimate of 45 to 90 minutes for the final cutover.
Compute first, data second. In production, Phase 1 shifted live traffic from the AWS VMs to the GKE load balancer via DNS while all reads and writes continued against AWS RDS over the VPN. Phase 2 enabled logical replication on RDS and ran a continuous Database Migration Service job into Cloud SQL, keeping the two databases in near-real-time sync.
A rehearsed cold cutover. Phase 3 was a stop-and-go cutover inside the planned window: drain traffic at the GKE ingress, quiesce the source database, promote the Cloud SQL replica once DMS reported no lag, repoint GKE configuration and secrets, and redeploy. A war room voice bridge kept migration, DevOps, development, and QA leads on one channel, and traffic reopened only after internal health checks passed against the new database.

The Results

Under 5 minutes of downtime. Against a planned 45 to 90 minute maintenance window, actual user-facing downtime at the production cutover was less than five minutes.
Zero data loss. Continuous DMS replication and the no-lag promotion gate meant the cutover moved the database without losing a single write.
A cloud-native production platform. The web application now runs containerized on GKE, state lives in managed services, and the conferencing fleet scales behind autoscaling groups instead of hand-tended VMs.
A proving ground that outlives the project. The rehearsal environment was built to outlast the migration: a permanent, Terraform-defined staging environment where future releases, upgrades, and scaling decisions get validated before they reach customers.
Everything as code, owned by the client. All infrastructure is defined in Terraform in 11Sight-owned repositories, so there was nothing to hand over at close-out that 11Sight did not already control.

This is the type of work our Accelerate solution delivers on, with foundations laid by Enclave: production capability built inside the client's environment, jointly with their team. Contact us to scope a similar migration.

Regulatory Evidence at Machine Speed

Andrew Stevens — Mon, 13 Jul 2026 09:38:21 +0000

The request arrived on a Tuesday and gave the compliance team twenty-four hours. The regulator wanted to know why the bank's transaction monitoring system had cleared a particular payment eleven months earlier: which rules fired and which did not, what customer risk score applied at that moment, who reviewed the resulting alert, and what that reviewer actually saw. The system that made the decision was still running, unchanged, and working correctly. The evidence about that one decision was somewhere else entirely. It sat in a queue, behind a request to rebuild a dataset, behind a restore from backup, behind a data engineer who had other work booked. The bank was not being accused of anything. It was being asked to show its working, and it had twenty-four hours to discover whether it could.

Banks have always been asked for evidence. What has changed is the nature of the request. The old rhythm was periodic, predictable, and aggregate: a return filed on schedule, a report at quarter end, a sample pulled for an inspection booked weeks in advance. The new rhythm is specific, unscheduled, and granular: this decision, this customer, this moment, and show me now. Most banks built their evidence systems for the first rhythm and are now being asked to serve the second. This is the banking form of a tension that runs through most regulated industries, which the wider series has set out as the gap between evidence and speed (see Evidence Versus Speed). Inside a bank it takes a particular shape, and it has a particular fix.

The twenty-four-hour request

What the compliance team actually had to do, in those twenty-four hours, is the most revealing part of the story. The decision they needed to explain was not recorded anywhere as a decision. It had to be reassembled from three systems that had never been designed to be read together.

The monitoring engine's logs held which rules had evaluated the payment, but those logs rotated after ninety days, so the relevant window had to be restored from backup. The customer risk score was worse. It was recomputed nightly and overwritten each time, so the score that actually applied eleven months ago no longer existed anywhere; it had to be rebuilt by rerunning the scoring logic against archived inputs and hoping the logic had not changed in the meantime. The analyst's review sat in a case management tool with its own retention policy and no link back to the payment except a reference number typed in by hand.

The team got there, barely, and the answer was correct. The payment had been cleared for good reasons and the bank had done nothing wrong. The cost was two engineers for the better part of a week and an uncomfortable realisation in the room afterwards: nobody was confident they could do it again. The decision itself had been sound. What the bank could not do was demonstrate it on demand, and that is a different kind of risk from getting the decision wrong. Under MAS Notice 626, records must be retained and be retrievable in a usable form (Monetary Authority of Singapore, 2022). Retention that exists in principle but cannot be produced when the regulator asks is not really retention at all.

What regulators used to ask for

The reason banks are in this position is that their evidence systems were built, quite rationally, for the requests that used to arrive. Regulatory reporting was a calendar activity. Returns were filed monthly, quarterly, or annually. Inspections were scheduled. The unit of evidence was the aggregate: a capital position, an exposure summary, a count of alerts raised and cleared. Banks built reporting factories to serve that model, with data warehouses, reconciliation processes, sign-off workflows, and a small industry of controls wrapped around the production of the report.

Even the regulation that pushed hardest on data quality assumed this shape. The Basel Committee's principles for effective risk data aggregation and risk reporting, published in 2013, told banks to be able to aggregate risk data accurately and to trace it, and it remains the reference point for banking data lineage (Basel Committee on Banking Supervision, 2013). But the output it had in mind was still a report, produced on a cycle, for a supervisor who would read it later.

Under that model, assembling evidence retrospectively was a perfectly sensible strategy, because the ask was predictable and the deadline was known. Banking compliance became organised around producing documents on a schedule. Evidence was a product of the reporting cycle rather than a property of the transaction, and for a long time nobody had reason to notice the difference.

What they ask for now

The difference is now impossible to miss, because supervisors have stopped confining themselves to the aggregate. They ask about individual decisions, at short notice, and they expect the bank to trace one end to end.

The pressure is visible in the supervisors' own output. More than a decade after the Basel principles, the European Central Bank found it necessary to issue a guide pressing banks on effective risk data aggregation and reporting, precisely because so many still cannot demonstrate complete, end-to-end lineage across their data estate (European Central Bank, 2024). The Digital Operational Resilience Act, in application since January 2025, requires financial entities to maintain registers of their ICT arrangements and to evidence their operational resilience continuously rather than annually (European Parliament and Council, 2022). And the five-year retrievability standard in MAS Notice 626 is not satisfied by a backup tape that takes a week to read.

Put together, these describe a single shift. The deliverable has become the reconstructable chain behind any single decision the bank made, available on request, at something close to the speed the bank operates. The report still gets filed, but it is no longer the thing the supervisor is really testing. Real-time compliance is a slightly misleading phrase, because nobody expects the answer instantly. What is expected is that producing the answer is a query rather than an excavation. A financial services audit is becoming a series of specific questions with specific answers, and regulatory reporting, while it continues, is no longer the whole of the job.

Engineering evidence into the flow

Meeting that expectation is an engineering problem, and it has an engineering answer. The evidence has to be emitted by the system as it runs.

Look again at what those twenty-four hours actually required. Engineers excavated rotated logs from a backup and hoped the restore was complete. They reran scoring logic against archived inputs and hoped the logic had not drifted in eleven months. They tied a case file to a payment through a reference number somebody had typed in by hand. Every step was archaeology. Every step introduced a guess. What the bank finally handed the regulator was a well-argued account of the decision, assembled under deadline by people reconstructing their own system from its debris.

Engineer the evidence into the flow and the same request lands very differently. At the moment the decision is made, the system writes it down. It records the inputs that fed the rule, the version of the rule and the model that evaluated them, the score they produced, the identity of anyone who touched the alert, and the time it happened, all bound together by a single identifier. It preserves the risk score as it stood that day instead of overwriting it at midnight. It hash-links the record, so any later alteration shows. It makes the record addressable, so one payment resolves to one chain. One approach excavates. The other retrieves.

This is what engineered compliance means in a bank. The evidence becomes a by-product of operating the system, produced continuously whether or not anyone asks for it. It costs something to build. It costs considerably less than paying for reconstruction every time and never being certain the reconstruction will hold.

None of that capability lives in the monitoring engine. It lives one layer down, in the data foundation that carries lineage and preserved history underneath every transaction the bank processes, and that layer is precisely what Sakura's Data & AI practice builds inside regulated institutions. A bank standing on that foundation can move quickly: Xapo Bank reached production on a tightly governed stack in weeks rather than quarters, because the controls were engineered into the platform instead of being negotiated afresh for every release.

The audit cycle that disappears

The payoff goes well beyond surviving the next twenty-four-hour request. The audit cycle stops being an event in the bank's calendar at all.

When evidence is a running output, the request that consumed two engineers for a week becomes a query answered in minutes by someone in compliance who does not need to open a ticket. Audit-ready banking means the bank answers without a change freeze, without pulling engineers off delivery, and without the background dread that the answer might not be reproducible. Everything else the bank is trying to build keeps moving while the question is answered.

The banks that bolt evidence on afterwards pay for it twice. They pay once in reconstruction, and again in the drag on everything else, because a system whose evidence cannot be produced on demand cannot safely be changed quickly. Every release carries an unpriced risk of breaking a chain nobody can currently see. There is a further dividend, too: a bank that can prove exactly what its transaction monitoring did, and why, can afford to tune it more aggressively, because it can demonstrate the effect of the change rather than argue about it. Evidence engineered into the flow is what lets a bank keep moving while it answers.

The work does not end when the architecture is built. The chain has to hold through every release, every model change, and every new rule, and someone has to be able to prove it still does. Running that discipline inside a bank is what Sakura's GRC service is for.

References

Basel Committee on Banking Supervision, 2013. Principles for effective risk data aggregation and risk reporting. Bank for International Settlements, Basel. Available at: https://www.bis.org/publ/bcbs239.htm [Accessed 10 July 2026].

European Central Bank, 2024. Guide on effective risk data aggregation and risk reporting. ECB Banking Supervision, Frankfurt. Available at: https://www.bankingsupervision.europa.eu/ecb/pub/pdf/ssm.supervisory_guides240503_riskreporting.en.pdf [Accessed 10 July 2026].

European Parliament and Council, 2022. Regulation (EU) 2022/2554 of the European Parliament and of the Council of 14 December 2022 on digital operational resilience for the financial sector (Digital Operational Resilience Act). Official Journal of the European Union, L 333, 27 December, pp. 1-79. Available at: https://eur-lex.europa.eu/eli/reg/2022/2554/oj [Accessed 10 July 2026].

Monetary Authority of Singapore, 2022. MAS Notice 626: Notice to Banks on Prevention of Money Laundering and Countering the Financing of Terrorism. Monetary Authority of Singapore. Available at: https://www.mas.gov.sg/regulation/notices/notice-626 [Accessed 10 July 2026].