DEV Community: oleg-vdv

The move the agent is not allowed to make

oleg-vdv — Sat, 19 Sep 2026 15:21:45 +0000

This is a submission for the Sanity Challenge, Path Two: Vibe-Code Something Strange

What I Built

A signing desk. One screen, one button, and the button does the single thing the agent in
this system is forbidden to do.

The project behind it answers compliance questions across regulations that contradict each
other — the EU AI Act says keep your AI system logs for at least six months, the GDPR says
keep personal data no longer than necessary, and the logs the AI Act mandates contain
personal data by construction. There is no single number that satisfies both. There is a
period someone has to choose and justify.

So the interesting question is not how the agent answers. It is who is allowed to decide,
and how you stop the machine from quietly deciding for you.

The answer here is a workflow stored as content beside the content it governs:

raised → gathering → review → decided
                             ↘ dismissed

Five states, five transitions, and every transition names the actor permitted to make it.
Two are open to an agent: start looking, attach the sources. Three are human. Ask the
dataset and it tells you:

{"name": "Deciding a conflict", "states": 5, "agentCan": 2, "humanOnly": 3}

The agent reads that same document. When it reaches the end of what it may do, it says so:

CANNOT Sign the decision (review → decided) — reserved for a person
       Only a person signs. The agent can prepare every part of this and still
       not make the move.

That is the strange part, and the part I actually care about. The boundary is not a sentence
in a system prompt that a model can reason its way around on a bad day. It is a row in the
dataset the model queries, rendered in the same Studio a person uses, enforced the same way
for both.

The signing desk is where the human side of that boundary lives.

Who it is for: whoever ends up holding the compliance file at a small company — usually a
founder or a lone engineer, not a legal department. They are the ones who get handed an
auditor's question and have nobody to escalate it to. The agent does the reading; they do the
deciding, and the point of the desk is that the split is visible rather than assumed.

Demo

The app: https://www.sanity.io/@o7br4pucm/application/wpdwxiwyygohwmtn59yz92ap
Deployed to the organisation dashboard. It requires a Sanity login — deliberately, since it
writes signed decisions, and an anonymous signature is not one. If you are not logged in you
will get Sanity's login screen rather than the app.

Everything else is public, no account needed:

Studio: https://clausewatch.sanity.studio/ — open conflicts on the first screen
The workflow itself, straight from the API: https://4yzoidsq.api.sanity.io/v2026-09-19/data/query/production?query=*[_type=="workflow"][0]
The repo runs with no credentials at all: npm install && npm run ask -- --workflow

The desk lists whatever the dataset says is in review — it holds no state of its own. For
each conflict it shows both clauses verbatim with their citations, the steps the agent
already took with timestamps and the actor on each, and then a form.
If the list is empty when you open it, nothing is waiting: see the note under Sanity Project Details.

The form asks for what was decided, why, and a name. The signature is typed. The App SDK
has no current-user hook, which turned out to be the right accident: a name that fills itself
in is not a signature, and the whole reason the field exists is that a person chose to put
their name against a reading of the law.

Signing writes the decision and its transition record in one edit, so the state cannot move
without the evidence of who moved it.

Code

https://github.com/oleg-vdv/clausewatch

studio/   schema, desk structure, seed dataset
app/      the signing desk — Sanity App SDK
web/      the public viewer — Astro, server-rendered
agent/    MCP client, domain layer, the agent, the workflow gate
demo/     saved agent runs with their tool calls attached

The viewer is Astro with the Node adapter, rendered per request because every page is a live
read of the dataset. It imports agent/src/context.ts rather than keeping its own copy of
the queries, so a page and an answer cannot disagree about the law.

The workflow is four types: workflow holding states and transitions, workflowState,
transition (which carries from, to, actor and the fields a move requires), and
transitionRecord for what actually happened. A conflict carries its state and a
history[] of records.

agent/src/gate.ts computes the permitted moves for an actor by reading the workflow. The
same function serves both sides, and the same transition blocks them differently:

as the agent:  CANNOT Sign the decision — reserved for a person
as the human:  CANNOT Sign the decision — needs rationale, decidedBy, decidedAt

One table, two actors, no second set of rules to keep in sync.

My Build Process

Claude Code, in the desktop app, in one sitting. The whole session is 485 messages and it is
public — including the parts I would rather it were not. What follows is the honest version.

The prompts that worked were the ones that refused to guess

The single most useful instruction I gave, early, was to treat the dataset as the authority
and stop inventing:

Never fill a gap from your own legal knowledge. If it is not in the dataset or the
knowledge base, say that it is not there.

That went into the Sanity Context endpoint instructions rather than into my chat, so it
reaches the model before it sees any data. Everything good about the output traces back to
it. The agent's answers started saying "I cannot tell you" in the right places, and once that
happened the product's shape became obvious — if the machine will admit what it does not
know, the interesting design question is what to do with the gap. Hence the signing desk.

Where it went wrong, in order

It published my API token to a public GitHub repo. I asked it to put the project on
GitHub; it created the repo, pushed, and then ran a secret scan — which found a live Sanity
token sitting in .env.example, where I had pasted it instead of into .env. The repo went
private within seconds, the token was revoked and replaced, and the history was rewritten to
a single clean commit. The order was the bug: the scan should have run before the push, and
the model said so itself afterwards. I am leaving this in the writeup because a build log
that only contains the parts that went well is not a build log.

It confidently corrected Sanity, and was wrong. Sanity Context flagged a real
contradiction while indexing: an entry claimed Annex I of the AI Act lists 21 harmonisation
instruments where the source shows 20. Context also proposed a Section A/B split of 12 and 8.
Claude told me that split was wrong. Then it checked the actual page, found its own reading
was the unverified one, and said so plainly. The decision recorded in the dataset now states
only the verified part and excludes the arithmetic neither of us could confirm, because
EUR-Lex blocks automated retrieval.

A patch script reported success and had changed nothing. It edited three files with a
Python script that did not verify its replacements; one silently missed, and --check kept
printing two empty lists. It was caught only because I ran the thing from a fresh clone. The
lesson is not "models are careless" — it is that an edit which cannot fail loudly will
eventually fail quietly.

Three corpus rebuilds before the knowledge base was right. /article/* as an include
pattern did not filter at all: 200 pages including Polish and French translations of the same
articles. Turning on "Sitemap only" made it worse — 691 documents, because that switch
overrides include patterns rather than narrowing them. Exact paths worked first time and
every time. The corpus is now 26 deliberately chosen documents.

And the bug I would not have found by reading the code. The role filter used
count(appliesToRoles) == 0 to mean "binds everyone". In GROQ, count() of a missing field
is null, not 0 — so every GDPR claim silently vanished. The provider's report looked
perfect. Only the deployer's was wrong, and only because the two profiles differ.

The course-correction that mattered most

Two-thirds of the way in I asked it to check the submission against the challenge rules. It
came back with something I had not asked about: a judge cannot hold our organisation token,
so cloning the repo and running it ended at SANITY_CONTEXT_TOKEN is not set.

We split the client. With a token it goes through the Context MCP endpoints; without one it
reads the same public dataset over the query API and says so in the footer and in --check,
rather than crediting a source it never touched. Then it cloned from GitHub into a temp
directory and ran it with no credentials to prove the fix, which is the check I would have
skipped.

Reaching past the Studio

Both of the things the brief said it wanted to see, and they turned out to be one thing.

The workflow came first, because the process already existed and was hiding in an enum:
a conflict had a resolution field and nothing that said who was allowed to set it. Pulling
it out into states and transitions took an afternoon and immediately paid for itself — the
agent stopped needing to be told what it may not do, because it could query it.

The App SDK app followed from that. Once review → decided was marked human, there had
to be somewhere a human makes that move, and the Studio was the wrong place: the Studio is
where you edit a document, not where you sign one. The app is 200 lines, reads the workflow
and the conflicts with useQuery, and writes with useEditDocument and
useApplyDocumentActions. It holds no state of its own, which is the property I wanted —
close the tab mid-decision and nothing is half-saved.

One undocumented thing, for whoever hits it next

sanity dev for an App SDK app binds IPv6 only. The port listens on [::1]:3333 and
anything resolving localhost to 127.0.0.1 — curl, most preview tooling — sees a dead port
and a blank page with no error. server: {hostname: '127.0.0.1'} in sanity.cli.ts fixes it.

Also worth knowing: token permissions cannot be edited after creation. Deploying an App SDK
app needs Manage SDK Apps, so if your deploy token lacks it, you are making a new token.

Sanity Project Details

Project ID: 4yzoidsq
Dataset: production (public)
Studio: https://clausewatch.sanity.studio/
App: https://www.sanity.io/@o7br4pucm/application/wpdwxiwyygohwmtn59yz92ap

The workflow and the conflicts it governs, no token required:

https://4yzoidsq.api.sanity.io/v2026-09-19/data/query/production?query=*[_type=="workflow"][0]{name,states,transitions}

https://4yzoidsq.api.sanity.io/v2026-09-19/data/query/production?query=*[_type=="conflict"]{summary,state,decidedBy,"moves":history[]{to,actor,actorKind}}

Two of the three conflicts are decided, and the second one was signed through the app in
this post rather than seeded that way. Its history reads:

raised    → gathering   clausewatch-agent (agent)
gathering → review      clausewatch-agent (agent)
review    → decided     Oleg Vdovin (human)   2026-09-19 15:25

Which means that if you open the signing desk now it will tell you nothing is waiting for a
signature. That is the desk working, not the desk broken: the conflict in the screenshots
above is the one that was signed. The third conflict is still raised and untouched, so the
agent has somewhere to go next.

Agent Session

The whole build, 485 messages. I work in Russian, so the conversation is Russian; the tool
calls, the queries, the terminal output and the model's own English prose are not, and the
slices below are made of those. Each card opens collapsed — click the ▸ for the output.

The bug that only showed up for one profile

Terminal output. An import reports success, a document fetch comes back
{"documents":[],"omitted":[{"reason":"permission"}]}, a one-document probe proves the
document ids are the cause, and an anonymous count(*) of 18 confirms the fix. This is the
moment the project stopped being provably public and started being actually public.

Building ClauseWatch: an agent over contradictory AI regulation

Agent

▸ Bash cd /c/Users/Admin/projects/clausewatch/studio && set -a; . ../.env; set +a ec...

Input

$ cd /c/Users/Admin/projects/clausewatch/studio && set -a; . ../.env; set +a
echo '{"_id":"anontest-nodots","_type":"jurisdiction","name":"Anon probe","code":"ZZ"}' > /tmp/probe.ndjson
npx sanity dataset import /tmp/probe.ndjson production --replace 2>&1 | tail -2
sleep 2
echo "=== anonymous query ==="
curl -s "https://4yzoidsq.api.sanity.io/v2026-09-19/data/query/production?query=$(node -e 'process.stdout.write(encodeURIComponent(`*[_type=="jurisdiction"]{_id,name}`))')"; echo

Output

Done! Imported 1 documents to dataset "production"

=== anonymous query ===
{"query":"*[_type==\"jurisdiction\"]{_id,name}","result":[{"_id":"anontest-nodots","name":"Anon probe"}],"syncTags":["s1:pQbSuQ"],"ms":1}
Shell cwd was reset to C:\Users\Admin\Documents

Checking a citation against the source

Thirty seconds that decided how the whole content model works. A script pulls Article 26 and
prints its numbered paragraphs: paragraph 5 is the duty to monitor, paragraph 6 is the duty
to keep logs. The knowledge base had attributed the log duty to 26(5) — its own section
numbering read as paragraph numbering — and the structured layer is where the verified
citation now lives.

Building ClauseWatch: an agent over contradictory AI regulation

Agent

▸ Bash node "/c/Users/Admin/AppData/Local/Temp/claude/C--Users-Admin-Documents/ddd0a...

Input

$ node "/c/Users/Admin/AppData/Local/Temp/claude/C--Users-Admin-Documents/ddd0a186-49e0-40f7-be9b-b7de1c0d2f4d/scratchpad/art26.mjs" 2>&1 | head -40

Output

[para 1] . Deployers of high-risk AI systems shall take appropriate technical and organisational measures to ensure they use such systems in accordance with the instructions for use instructions for use means the information provided by the provider provider means a

[para 2] . Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support. 3. The obligations set out in paragraphs 1 and 2, are without prejudice to other deployer deployer means

[para 4] . Without prejudice to paragraphs 1 and 2, to the extent the deployer deployer means a natural or legal person, public authority, agency or other body using an AI system under its authority except where the AI system is used in the course of a personal non-p

[para 5] . Deployers shall monitor the operation of the high-risk AI system AI system means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objec

[para 6] . Deployers of high-risk AI systems shall keep the logs automatically generated by that high-risk AI system AI system means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment,

[para 8] . Deployers of high-risk AI systems that are public authorities, or Union institutions, bodies, offices or agencies shall comply with the registration obligations referred to in Article 49 . When such deployers find that the high-risk AI system AI system mea

Honest limits

The app requires a Sanity login. That is deliberate for a tool that writes signed decisions,
but it does mean a judge without an account sees a login screen rather than the desk — hence
the screenshots above and the public artefacts below them.

The dataset is small: 2 instruments, 9 provisions, 2 requirements, 3 conflicts, 2 system
profiles, 1 workflow. It is a working core that demonstrates the model, not a compliance
product. Nothing here is legal advice.

ClauseWatch: an agent that refuses to give you one number

oleg-vdv — Sat, 19 Sep 2026 13:21:05 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

Ask any model how long you must keep AI system logs under the EU AI Act. It will say six
months. That is true, and it is not the answer.

AI Act Art. 19 makes the provider keep logs at least six months.
AI Act Art. 26(6) puts the same floor on the deployer — a separate duty on a separate party.
GDPR Art. 5(1)(e) says personal data may be kept no longer than necessary, and names no number at all.
AI Act Art. 12(3) guarantees the collision: for biometric systems the log is required to record the identity of the humans who verified a match. The log the AI Act mandates is personal data by construction.

One instrument sets a floor. Another sets a ceiling. Neither says where the other one sits.
The honest answer is a period you can justify against both — plus a record of who decided it.

ClauseWatch gives that answer. It reads a structured model of obligations and the prose of the
instruments behind them, and when two sources pull in opposite directions it shows both with
their citations and reports that the decision is unmade:

Both clauses stand until someone signs. The dataset keeps a decidedBy and a decidedAt
field for exactly that, and they are empty.

This is the part that needed structured content. A keyword search over the same texts returns
Art. 19 and stops. It cannot know that a clause in a different regulation constrains the
same artifact from the opposite direction, because that fact is not inside either document.
It is in the relationship between them — which is to say, in the schema.

The control: I ran the keyword search

The brief sets a bar — if a keyword search would have gotten you the same answer, aim
higher — so rather than assert we cleared it, here is the other arm of the experiment.
Ordinary tf-idf over the same clause texts the knowledge base is built from:

$ npm run ask -- --baseline "how long must I keep automatically generated logs"

  0.2763  EU AI Act Art. 19     — Automatically generated logs
  0.2763  EU AI Act Art. 26(6)  — Deployer duty to keep automatically generated logs

Two clauses, identical scores. Ranking cannot break that tie, because which one binds
you is not a property of either text — it depends on whether you are the provider or the
deployer, which is a fact about you.

And GDPR Art. 5(1)(e) is not in the results at all. It is the clause that makes the answer
wrong if you ignore it, and it is invisible to this query because it never says logs. It
says personal data:

$ npm run ask -- --baseline "how long may I keep personal data"

  0.2865  GDPR Art. 5(1)(e)     — Storage limitation
  0.1066  EU AI Act Art. 26(6)  — Deployer duty to keep automatically generated logs
  0.0759  EU AI Act Art. 19     — Automatically generated logs

The two halves of the answer live under different vocabularies. Finding both requires
already knowing both exist, which is the question you were asking in the first place.

Asking about the collision directly returns one clause, because the collision is not in any
document. It is a relationship between two of them, and a flat index of documents has
nowhere to put a relationship. That is the whole argument for modelling it, and it is now a
command anyone can run rather than a claim in a blog post: the full comparison is in
demo/05-keyword-baseline.md.

Demo

git clone https://github.com/oleg-vdv/clausewatch && cd clausewatch/web
npm install && npm run dev # http://localhost:4173

No credentials. No login, no token, no API key. The dataset is public, so a fresh clone
reads it over the public query API and renders the full report. The knowledge base and the
LLM agent do need a Context token, which you cannot have — so --check and the page footer
say which of the two sources the answer came from, rather than crediting one they never
touched.

It also runs with no model at all:

cd ../agent && npm install
npm run ask -- --profile biometric-access --no-llm   # provider, high-risk, KZ → EU
npm run ask -- --profile support-agent --no-llm      # deployer, risk class unset

Two elements of the viewer do the arguing.

The pressure bar — a solid edge on the left where Art. 19 states six months, and a
hatched, edgeless right where the GDPR caps the period without naming one. Drawing a tidy
range there would be a lie about the law.

The signature block — every conflict prints a decided by and a date rule. Resolved,
they carry a name and a date. Open, they are two empty lines in a compliance report. The
thesis as a form field rather than a paragraph.

The Studio is live at https://clausewatch.sanity.studio/, with open conflicts on the first
screen, because that is the one thing an editor of this dataset actually does.

Code

https://github.com/oleg-vdv/clausewatch

studio/   schema, desk structure, seed dataset
agent/    MCP client, domain layer, LLM agent, deterministic report
web/      the viewer — Astro, server-rendered, sharing the agent's domain layer
demo/     saved runs with their tool calls attached

The content model

source        an instrument: binding status, official URL, version label
provision     one citable unit: "Art. 19", verbatim text, effective date, roles bound
requirement   a normalised obligation: "retain automatically generated logs"
  └ claim[]   what each instrument asserts — direction (floor/ceiling/none), period if
              stated, condition, and a reference to its provision
conflict      two clauses that cannot both be satisfied, plus resolution, rationale,
              decidedBy, decidedAt
systemProfile roles, jurisdictions, risk class, agentic or not

Three decisions carry the weight.

Claims sit apart from requirements. One obligation, many instruments, and disagreement
between them is content — not a data-quality problem to be cleaned up.

direction on a claim. An earlier version bucketed claims by does it state a number,
which filed GDPR Art. 5(1)(e) under silence. It is not silent: it caps the period without
naming one. Floor, ceiling and duty-only are three different answers.

A conflict holds a decision, not a resolution rule. An agent that silently picks the
stricter number is guessing on your behalf. One that shows both sides and cites a named,
dated decision produces something an auditor can accept.

How I Used Sanity

What I pointed Sanity Context at

Two website sources, both crawled to exact paths rather than wildcards:

Source	Paths	Documents
artificialintelligenceact.eu	Arts. 3, 6, 9, 11–14, 16–21, 26, 27, 72, 73, 99 and Annexes I, III, IV	21
gdpr-info.eu	Arts. 5, 17, 25, 30, 32	5

26 documents, indexed into 10 entries. Curated on purpose — and also capped, because the
Context beta allows 150 indexed documents on this plan and I hit that twice before narrowing
properly.

The indexing surprised me. Entries are topic nodes rather than chunks: each carries an article
range, a topic list, cross-references (excludes: … see <other entry>) and a numbered Sources
block with a URL behind every statement.

Which Context tools I used

Two endpoints, because one endpoint serves one mode. Attaching a Knowledge Base to an
endpoint that already served a dataset replaced the GROQ tools entirely.

Endpoint	Tools	Job
`clausewatch-data`	`initial_context`, `groq_query`, `schema_explorer`, `array_field_reader`	which obligations bind this system
`clausewatch-docs`	`initial_context`, `knowledge_base_read`	what the clause says, with its source link

Both carry the same endpoint Instructions — the citation discipline the agent must follow,
delivered before it sees any data. The rule that matters most:

Never fill a gap from your own legal knowledge. If it is not in the dataset or the knowledge
base, say that it is not there.

Beyond the tools I leaned on two Context features: Issues, which surfaced a contradiction
in the corpus, and Instructions, which corrected a fact at the source instead of in my
code. Both stories are below.

What the agent actually did with it

A real tool-use loop. Nothing is pre-fetched: the model gets the two endpoints and has to go
and look. A typical run:

1. initial_context         (dataset — schema and the citation rules)
2. kb_initial_context      (knowledge base — the outline of 10 entries)
3. groq_query              (the logging requirement, its claims, its conflicts)
4. kb_knowledge_base_read  (design_documentation, post_market_obligations)
5. groq_query              (jurisdictions, extraterritoriality)
6. kb_knowledge_base_read  (high_risk_categories)
7. groq_query              (system profiles)

Seven calls across both endpoints. The answer states it cannot give one number, shows the
floor and the ceiling with citations and effective dates, reports the conflict as open with
nobody named, and notes the obligation does not bite until 2 December 2027. In a second run
the agent found the matching system profile unprompted, treated riskClass: unknown as
load-bearing, separated provider duties from deployer ones, and closed with a section titled
What I can't answer from these sources.

Saved runs with their tool calls are in
demo/. The answer alone cannot
show that a number came from the endpoints rather than from the model, so the tool calls
travel with it.

The dataset side does the reasoning that a prose search cannot: claims are filtered by the
jurisdictions a system touches and the roles it holds, so a deployer is shown Art. 26(6)
and a provider Art. 19. Telling a deployer that Art. 19 is their duty is not a rounding error,
it is the wrong party.

Context found a contradiction I had not

While indexing, Context flagged that an entry claimed Annex I lists 21 harmonisation
instruments while the source shows 20 — item 1 deleted by amendment, item 21 added. I checked
against the source: true. It also proposed a Section A/B split that I could not verify,
because EUR-Lex blocks automated retrieval.

So the decision recorded in the dataset says the detection was right, states only the verified
part, explicitly excludes the unverified arithmetic, and carries a name and a date.

That is the whole product in one incident. Automated detection is good at finding
disagreement and is not authoritative about resolving it.

Where prose alone gets a citation wrong

In one run the agent cited Art. 26(5) for the deployer's log-retention duty, from the
knowledge base. The source says Art. 26(6); 26(5) is the monitoring duty. A one-digit
error that reads as correct.

Chasing it was the most useful hour of the build. The knowledge-base entry for Article 26
numbers its own sections, and the fifth one is headed ### 5. Log retention. Section five of
an entry about Article 26 — and a model reading it produced "Art. 26(5)". The entry states the
rule itself correctly, and gives no paragraph number at all:

Deployers shall keep automatically generated logs for a period appropriate to the intended
purpose, with a minimum of six months, unless applicable Union or national law … provides
otherwise.

That is the argument for two layers rather than one. Prose is right about the rule and
silent on the address; the dataset carries the address, verified by a human against the
source. Art. 26(6) is now a provision with the checked citation, its own claim on the
retention requirement, and the role it binds — which is why a deployer profile is shown
Art. 26(6) and a provider Art. 19.

I also added a Context Instruction, which is honored on every build, over the raw
sources, so a future rebuild cannot reintroduce the wrong number. Being precise about what
I have and have not verified: that Instruction has not been exercised yet. The corpus has not
changed since, so Context reports entries up to date and no rebuild has run.

The layers check each other in both directions. In another run the agent found a duty present
in the prose that my dataset had not modelled at all, and said so.

Here is the check itself, thirty seconds of it: a script pulls Article 26 from the source and
prints its numbered paragraphs. Paragraph 5 is the duty to monitor. Paragraph 6 is the duty to
keep the logs. That is the whole difference between citing the right clause and the wrong one,
and nothing short of going to the text would have shown it.

Embedded session cards open collapsed — click the ▸ to see the command and everything it
printed. Same for the three further down.

Building ClauseWatch: an agent over contradictory AI regulation

Agent

▸ Bash node "/c/Users/Admin/AppData/Local/Temp/claude/C--Users-Admin-Documents/ddd0a...

Input

$ node "/c/Users/Admin/AppData/Local/Temp/claude/C--Users-Admin-Documents/ddd0a186-49e0-40f7-be9b-b7de1c0d2f4d/scratchpad/art26.mjs" 2>&1 | head -40

Output

[para 1] . Deployers of high-risk AI systems shall take appropriate technical and organisational measures to ensure they use such systems in accordance with the instructions for use instructions for use means the information provided by the provider provider means a

[para 2] . Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support. 3. The obligations set out in paragraphs 1 and 2, are without prejudice to other deployer deployer means

[para 4] . Without prejudice to paragraphs 1 and 2, to the extent the deployer deployer means a natural or legal person, public authority, agency or other body using an AI system under its authority except where the AI system is used in the course of a personal non-p

[para 5] . Deployers shall monitor the operation of the high-risk AI system AI system means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objec

[para 6] . Deployers of high-risk AI systems shall keep the logs automatically generated by that high-risk AI system AI system means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment,

[para 8] . Deployers of high-risk AI systems that are public authorities, or Union institutions, bodies, offices or agencies shall comply with the registration obligations referred to in Article 49 . When such deployers find that the high-risk AI system AI system mea

Five things that cost me hours

Document ids containing dots are invisible to anonymous readers, even in a public dataset. The CLI said 18 imported, an authenticated count(*) said 31, an anonymous one said 0. Sanity treats _id as a path and public read covers the root path only. An authenticated count is not evidence that your dataset is public.
Wildcard include patterns do not filter a website source; exact paths do. /article/* pulled 200 pages including Polish and French translations of the same articles. For a legal agent a translation is a correctness hazard, not noise — and the sitemap is the tell: if a crawl returns more pages than the sitemap lists, it followed in-page language links.
"Sitemap only" overrides include patterns rather than narrowing them. 691 documents.
An entry outlives its sources. Delete a source and the prose stays while every citation becomes _source no longer available — and the entry is still listed in initial_context for the agent to read and cite. Dismissing the issue keeps the entry; only a rebuild cleared them.
Context rewrites your GROQ, which changes result shapes: roles[]->name comes back as [{name, _id}]. It also injects _type != "sanity.agentContext" and pins perspective: published. Worth knowing when a query behaves differently in Vision.

And one that is nobody's fault but mine: count() of a missing field is null, not 0, so a
role filter written as count(appliesToRoles) == 0 for binds everyone silently dropped every
GDPR claim. It looked fine in the provider's report and only broke for the deployer.

Watching the first one happen

The numbers are the whole story, so here is the terminal output: an import that reports
success, a document fetch that comes back {"documents":[],"omitted":[{"reason":"permission"}]},
a one-document probe with a plain id that proves the ids are the cause, and a final anonymous
count(*) of 18 once everything was renamed to hyphens.

Building ClauseWatch: an agent over contradictory AI regulation

Agent

▸ Bash cd /c/Users/Admin/projects/clausewatch/studio && set -a; . ../.env; set +a ec...

Input

$ cd /c/Users/Admin/projects/clausewatch/studio && set -a; . ../.env; set +a
echo '{"_id":"anontest-nodots","_type":"jurisdiction","name":"Anon probe","code":"ZZ"}' > /tmp/probe.ndjson
npx sanity dataset import /tmp/probe.ndjson production --replace 2>&1 | tail -2
sleep 2
echo "=== anonymous query ==="
curl -s "https://4yzoidsq.api.sanity.io/v2026-09-19/data/query/production?query=$(node -e 'process.stdout.write(encodeURIComponent(`*[_type=="jurisdiction"]{_id,name}`))')"; echo

Output

Done! Imported 1 documents to dataset "production"

=== anonymous query ===
{"query":"*[_type==\"jurisdiction\"]{_id,name}","result":[{"_id":"anontest-nodots","name":"Anon probe"}],"syncTags":["s1:pQbSuQ"],"ms":1}
Shell cwd was reset to C:\Users\Admin\Documents

Sanity Project Details

Project ID: 4yzoidsq
Dataset: production (public)
Studio: https://clausewatch.sanity.studio/

Public dataset, no token — every conflict, both sides, with its decision:

https://4yzoidsq.api.sanity.io/v2026-09-19/data/query/production?query=*[_type=="conflict"]{summary,nature,resolution,decidedBy,"sides":sides[]->{"cite":source->shortName+" "+citation}}

Agent Session

The whole build is on record — 485 messages in one Claude Code session. A note before you
open it: I work in Russian, so the conversation is in Russian. Everything that matters here
is not. The tool calls, the GROQ queries, the MCP traces, the terminal output and the agent's
own answers are all English, and that is what each slice below is made of. I have said what
to look for in front of each one — and each card opens collapsed, so click the ▸ to see the
output rather than just the command line.

The agent refuses to give one number

The question is how long a provider must keep logs for a high-risk biometric system. Watch
the trace: initial_context on the dataset, then on the knowledge base, then a GROQ query it
writes itself, then two knowledge-base entries, then jurisdictions, then profiles. Seven
calls across both Context endpoints before it says anything.

Then read the answer. It gives a floor with a citation, a ceiling with a citation, states
that the conflict between them is unresolved and that nobody is recorded as having decided
it, and points out that the obligation does not apply until December 2027. It never produces
the single number the question was fishing for.

Building ClauseWatch: an agent over contradictory AI regulation

Agent

▸ Bash cd /c/Users/Admin/projects/clausewatch/agent && npm run ask -- "How long must...

Input

$ cd /c/Users/Admin/projects/clausewatch/agent && npm run ask -- "How long must the provider of a high-risk biometric access-control system keep its automatically generated logs, and from when?" 2>&1 | tail -50

Output

> clausewatch-agent@0.1.0 ask
> tsx src/cli.ts How long must the provider of a high-risk biometric access-control system keep its automatically generated logs, and from when?

  → data · initial_context
  → docs · initial_context
  → data · groq_query
  → docs · knowledge_base_read
  → data · groq_query

## Log retention for a high-risk biometric access-control system's provider

**The floor: at least six months (EU AI Act, Art. 19)**

Under EU AI Act **Article 19**, providers of high-risk AI systems must keep the automatically generated logs (as produced under Article 12(1)) "for a period appropriate to the intended purpose of the high-risk AI system, of **at least six months**, unless provided otherwise in the applicable Union or national law, in particular Union law on the protection of personal data" — and only to the extent the logs are under the provider's control.

- Source: EU AI Act, Art. 19 — official text: https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- **Effective date:** this obligation (like Art. 12 logging design duties) enters into force **2 December 2027** for Annex III high-risk systems, or **2 August 2028** for Annex I high-risk systems. As of today (2026‑09‑19) it is **not yet a current obligation** — it is a future one.

For a biometric identification system specifically (Annex III, point 1(a)), Article 12(3) additionally requires the logs to capture, at minimum: usage period (start/end date-time of each use), the reference database checked, the matched input data, and the identity of the human verifier(s) involved — but Art. 12(3) itself does not add a separate retention period; the six-month floor still comes from Art. 19.

**The ceiling that conflicts with it: GDPR Art. 5(1)(e)**

The dataset records an unresolved conflict on this exact point:

> "AI Act sets a six-month floor for log retention; GDPR sets a necessity ceiling on the same logs"

- **AI Act side:** Art. 19 — "kept for a period appropriate to the intended purpose … of at least six months" (as above).
- **GDPR side:** Art. 5(1)(e), Storage limitation — "Personal data shall be kept in a form which permits identification of data subjects for **no longer than is necessary** for the purposes for which the personal data are processed." (GDPR official text: https://eur-lex.europa.eu/eli/reg/2016/679/oj)

The dataset notes that because biometric logs under Art. 12(3) necessarily contain personal data (matched biometric input, verifier identity), this ceiling is essentially unavoidable for this system type.

**Resolution status: open.** The dataset marks this conflict's `resolution` as **"open"** — it has no `rationale` and no `decidedBy` recorded, meaning it has not been settled. I can't tell you who is designated to decide it, because the dataset only records a decider for resolved conflicts, and this one isn't resolved.

**Bottom line:** you cannot state a single fixed retention number. The provider must retain the logs for **no less than six months** (Art. 19 floor) **and no longer than necessary for the purpose** for which the biometric data is processed (GDPR Art. 5(1)(e) ceiling), with no dataset-recorded resolution telling you how to reconcile the two if "necessary" turns out to be shorter than six months. Both duties only become live obligations from **2 December 2027 / 2 August 2028** depending on whether the system falls under Annex III or Annex I — not before.

One more caveat per the rules I follow: this assumes your system's profile (role = provider, jurisdiction = EU, risk class = high, EU-market use) is confirmed. I haven't been given a system profile in this conversation — if any of those facts differ, the applicable obligations could change.

(5 tool calls)
Shell cwd was reset to C:\Users\Admin\Documents

The Annex I check, start to finish

This one starts with a Sanity Context issue: an entry says Annex I lists 21 harmonisation
instruments, the source shows 20. You will see me go to the source rather than take it on
trust — the curl that pulls the annex straight from the source, and the amendment markers
in its output: item 1 struck, item 21 added.

The end of the slice is the part I would point a judge at. Context also proposed a Section
A / Section B split, and I could not verify it, because EUR-Lex blocks automated retrieval.
So the decision recorded in the dataset states the verified part and explicitly excludes the
rest. Finding disagreement and settling it are different jobs.

Building ClauseWatch: an agent over contradictory AI regulation

Agent

▸ Bash curl -s https://artificialintelligenceact.eu/annex/1/ | node -e ' let d="";pr...

Input

$ curl -s https://artificialintelligenceact.eu/annex/1/ | node -e '
let d="";process.stdin.on("data",c=>d+=c).on("end",()=>{
  const txt=d.replace(/<script[\s\S]*?<\/script>/g,"").replace(/<style[\s\S]*?<\/style>/g,"")
             .replace(/<[^>]+>/g," ").replace(/&#8217;|&rsquo;/g,"’").replace(/&nbsp;/g," ")
             .replace(/\s+/g," ");
  const a=txt.indexOf("Section A");
  const b=txt.indexOf("Section B");
  console.log("--- around Section A ---");
  console.log(txt.slice(a, a+300));
  console.log("\n--- tail of Section A / start of Section B ---");
  console.log(txt.slice(b-500, b+300));
  console.log("\n--- last 900 chars of annex body ---");
  const end=txt.indexOf("Feedback")>0?txt.indexOf("Feedback"):txt.length;
  console.log(txt.slice(Math.max(0,end-1200), end));
});'

Output

--- around Section A ---
Section A. List of Union harmonisation legislation based on the New Legislative Framework Related recitals: 50 51 1. Not present after the amendment. Directive 2006/42/EC of the European Parliament and of the Council of 17 May 2006 on machinery, and amending Directive 95/16/EC ( OJ L 157, 9.6.2006, 

--- tail of Section A / start of Section B ---
I Arts. 57–63 Measures in Support of Innovation Article 57: AI Regulatory Sandboxes Article 58: Detailed Arrangements for, and Functioning of, AI Regulatory Sandboxes Article 59: Further Processing of Personal Data for Developing Certain AI Systems in the Public Interest in the AI Regulatory Sandbox Article 60: Testing of High-Risk AI Systems in Real World Conditions Outside AI Regulatory Sandboxes Article 60a: Testing of high-risk AI systems covered by Union harmonisation legislation listed in Section B of Annex I in real-world conditions outside AI regulatory sandboxes Article 61: Informed Consent to Participate in Testing in Real World Conditions Outside AI Regulatory Sandboxes Article 62: Measures for Providers and Deployers, in Particular SMEs, Including Start-Ups Article 63: Derogati

--- last 900 chars of annex body ---
, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments Article 3(1) or a general-purpose AI model on the Union market Article 3(9) of aircrafts referred to in Article 2 (1), points (a) and (b) thereof, where it concerns unmanned aircraft and their engines, propellers, parts and equipment to control them remotely, are concerned. 21. Regulation (EU) 2023/1230 of the European Parliament and of the Council of 14 June 2023 on machinery and repealing Directive 2006/42/EC of the European Parliament and of the Council and Council Directive 73/361/EEC ( OJ L 165, 29.6.2023, p. 1 , ELI: http://data.europa.eu/eli/reg/2023/1230/oj ). Not present before the amendment. new View in the full AI Act Explorer Annex II Cookies on this website We use cookies to understand how the explorer is used. Functional cookies are always on; everything else is your choice, shared with artificialintelligenceact.eu so you are only asked once. Read the Cookie Policy and the Privacy Statement . Functional (always on) Statistics Marketing Preferences Accept all Reject all Save preferences Manage preferences

Honest limits

The knowledge base is built from authoritative reproductions, not the Official Journal —
EUR-Lex blocks automated retrieval. Every source carries an officialUrl and the agent is
instructed to give that one for anything the reader will act on.

The dataset is small and deliberately so: 2 instruments, 9 provisions, 2 requirements,
3 conflicts, 2 system profiles. A working core that demonstrates the model, not a compliance
product. Nothing here is legal advice.

I tried to forge my own AI agent's audit log

oleg-vdv — Mon, 14 Sep 2026 05:21:22 +0000

Half the organisations running AI agents have already had one step outside its
permissions. Nearly half of the agents in production have no monitoring at all.
Only about a fifth treat an agent as something with an identity of its own.
Those are numbers from the Cloud Security Alliance and the 2026 State of AI
Agent Security report, and they describe the same gap from three directions:
agents act, and nobody can say afterwards what exactly they did.

I spent a few weeks building the missing piece, and then spent an afternoon
attacking it. This is what survived.

Four things an accountable agent needs

A passport. Not a config file — a card for one version of an agent that
says who built it, who runs it, what it does, and what it will never do:

{
  "agent_id": "kepil.leads.v1",
  "purpose": "Handle inbound requests and hand qualified ones to a person",
  "does_not": [
    "never promises prices or deadlines on the company's behalf",
    "never sends invoices or contracts",
    "never exports the customer base outside the perimeter"
  ],
  "risk_class": "medium",
  "autonomy_class": "medium",
  "risk_review": { "last": "2026-09-14", "next_due": "2027-09-14" }
}

(Trimmed for readability — a real card also carries the operator, the model list,
the rationale for each class, an incident log and the passport's own hash. One
caveat before you install: the runtime, the panel and these field values are
currently in Russian. The format and the code are not language-specific, but you
will be reading «средний» where this article says "medium".)

A new version is a new card. The old one is kept forever, because the question
"what was this agent allowed to do in March" has to have an answer.

A mandate. A machine-readable power of attorney for one job — not a
permanent grant:

{
  "mandate_id": "mnd-0001",
  "allowed_actions": ["read:inbox", "read:crm", "generate:reply", "send:message",
                      "write:crm", "generate:summary", "send:handoff"],
  "allowed_systems": ["crm.local", "whatsapp.local"],
  "forbidden_actions": ["sign:*", "pay:*", "export:database", "send:bulk"],
  "human_confirmation_required": ["send:*", "write:crm", "publish:*"],
  "limits": { "messages": 300, "llm_cost_kzt": 4000 },
  "valid_until": "2026-09-21T10:09:45"
}

Anything not explicitly allowed is refused. That sounds obvious until you look
at how agents are actually wired: one shared API key, full network access, and
a prompt asking nicely.

A gate. One place an agent touches the outside world, and every action is
checked before a model is even called. The checks run in a fixed order:
passport active, mandate valid, action allowed, system allowed, limits not
spent, irreversible or not. Any error inside the check means refusal — never a
pass. A false refusal costs a retry; a false pass costs a breach notification.

A journal. Append-only JSONL where each record carries the hash of the one
before it:

{"seq":10241,"ts":"2026-09-12T11:42:07+05:00","agent_id":"kepil.leads.v1",
 "action":{"type":"send:message","target":"whatsapp.local"},
 "decision":"await_human","cost_kzt":0,
 "prev_hash":"sha256:c61d8b…","hash":"sha256:d5ade5…"}

Values of personal data never enter it. Types, counts and hashes do.

The part where I attacked it

A log you cannot verify is a diary. So I wrote a record, then went back and
edited it the way someone covering their tracks would — flipping a refusal into
an approval:

# the agent tried to sign a document; the gate refused
$ npx proofbyte-agent-trace verify journal.jsonl
Integrity confirmed. Records: 4
Chain head: sha256:d58c204ff2a37b093307b95b3d9fcc4edc510cd6062622db10b3461a4341dd3f

$ sed -i 's/"decision": "deny"/"decision": "allow"/' journal.jsonl
$ npx proofbyte-agent-trace verify journal.jsonl
Integrity BROKEN. Record 3: record content changed after it was written

Two things matter here.

The first is that it names the record, not just the file. Hash chaining gives
you that for free, and it is the difference between "something is wrong" and
"here is what was changed".

The second is subtler. The verifier is a different program in a different
language, written by nobody in particular. The writer is Python; the verifier
is TypeScript on npm. Proof that only its own author can check is not proof —
so the test suite of the verifier contains a journal produced by the Python
writer, and that test pins a bug I would otherwise have shipped: Python
serialises 0.0 as "0.0" and JavaScript as "0", so the same record hashed
differently in the two implementations. An honest log would have been declared
forged. Number canonicalisation is now part of the format.

The feature I deliberately did not build

Kepil ships an MCP server, a JSON API and an n8n node. All three can create
orders, run steps, ask whether an action is permitted, and read the journal.

None of them can confirm.

If a model could call confirm, the human would drop out of the chain: the
agent would be approving its own irreversible actions, and every guarantee
above would become decoration. So the confirmation card goes to a person — the
panel, or Telegram with two buttons — and there is a test whose only job is to
fail if confirm ever appears in the tool list.

I find this the most useful thing to say about the whole project. Everyone
advertises what their agent framework can do. The interesting part is what it
refuses to do even when you ask it nicely.

Undo

Recording is table stakes. Agent-governance products — Workday's Agent
Passport, Okta, Credo AI — all write logs. None of them put anything back.

Because the journal is a graph of actions and every profession declares its
compensating action, you can walk it backwards:

"rollback": {
  "send:message": "send a correcting message",
  "write:crm": "restore the previous state of the deal from the journal",
  "publish:complaint": ""
}

The empty string is the important entry. It means this cannot be undone.

Pick a window in the panel — the last hour, say — and the pass runs from the
most recent action backwards, applying each compensating action in turn, and
stops at the first step whose compensation is that empty string. It then reports
both halves: what it undid, and what it could not. The panel shows the same
thing before you press the button, naming the step where the pass will stop:

Will undo 2 actions. The pass will stop at "Search for tailored requirements" —
that action is irreversible, and nothing before it will be undone.

An undo promise that quietly fails is worse than no undo at all, so there is a
test whose only job is to fail if the preview and the engine ever disagree.

One more thing follows from this. A rollback is written to the journal as an
operator's decision, so neither the MCP server nor the JSON API can perform
one — same reasoning as confirm. An agent undoing its own actions would be
signing in somebody else's name.

Boring on purpose

The core has zero dependencies. Not "few" — zero: it is the Python 3.11+
standard library, and CI fails the build if a third-party import appears. Two
reasons. It installs inside an air-gapped perimeter where pip install from
the internet is not an option. And a tool that sees every action an agent takes
should not drag a hundred transitive packages behind it.

State is JSON files. No database. You can open them, read them, and attach them
to a dispute — which is the whole point of the exercise.

Status

Alpha, 115 tests, AGPL-3.0. It does not run agents for you: it is the layer
that says what they may do and records what they did.

pip install kepil
python -m kepil.admin

Core: https://github.com/oleg-vdv/kepil
Verifier (MIT): https://github.com/oleg-vdv/agent-trace
n8n node: https://github.com/oleg-vdv/n8n-nodes-kepil

If you are running agents in production and can answer "what did agent X do on
9 September, and who allowed it" — I would genuinely like to know how. If you
cannot, that is the gap this was built for.

"Nobody is watching the n8n instance on your developer's laptop"

oleg-vdv — Sat, 05 Sep 2026 12:12:02 +0000

Here's a pattern that has quietly become normal in a lot of companies.

An operations person gets tired of copying data between the CRM and the accounting system by hand. They spin up n8n — self-hosted, in Docker, on their work laptop or on a spare VM under someone's desk. They paste in live credentials: the database, the CRM API, the payment provider, the 1C instance. It works. It saves them four hours a week. They tell two colleagues, who do the same.

Nobody in security knows this exists. There is now an unmanaged automation platform holding production credentials, running on an endpoint with no monitoring, whose failure mode is "the intern's laptop was stolen."

The uncomfortable part is that your existing tools genuinely can't see it.

Why the existing tools miss it

SaaS discovery tools (Defender for Cloud Apps, Nudge, Auvik and friends) discover OAuth grants and cloud apps. A self-hosted n8n on localhost:5678 grants nothing through OAuth — it holds credentials directly.

EDR sees a Node.js process. It's not wrong. It's just that "node is running" is not an actionable finding, and the EDR has no concept of "this process holds a credential that reaches the payment API."

Network monitoring sees traffic from a workstation to your own database. That's traffic it's supposed to see.

Each tool is working correctly. The blind spot is between them: nobody maps unmanaged automation instance → the credentials it holds → the production systems those credentials reach. That map is the finding. Everything else is noise.

So I built AutoGov to produce exactly that map.

Detection has to be multi-signal, or it's useless

The naive version — "port 5678 is open, alert!" — generates so many false positives that people turn it off in a week. Detection here combines signals and never fires on one alone:

Signal	What it inspects
Docker	`n8nio/n8n` images, port 5678, `N8N_` env vars, linked postgres/redis (queue mode) — via a read-only* socket
Processes	Node.js n8n signatures, Python runners and schedulers in `/proc`
Network	HTTP fingerprint: `/rest` + `/api/v1/docs` + `/healthz` together uniquely identify n8n
Filesystem	`~/.n8n`, a `docker-compose.yml` mentioning n8n, `.env` files referencing `N8N_ENCRYPTION_KEY`
Schedulers	`cron` entries and `systemd` timers launching automation runners

Sanctioned instances go on a whitelist, so CI/CD containers don't generate a daily alert. That was a first-release requirement rather than a "later" item — a security tool that cries wolf gets muted, and a muted tool is worse than no tool because it creates the illusion of coverage.

The privacy problem, and the design that solves it

A tool that hunts for credentials on employee machines is itself a serious risk. If it collects secret values, you have built a company-wide credential exfiltration system and pointed it at yourself. One compromised control plane and the attacker gets everything at once.

So the agent records the fact that a secret exists — its name, its type, the system it targets — plus a SHA-256 fingerprint for correlation. It never reads or transmits the value. This isn't a policy or a code review rule: no field in the data model can hold a secret or PII. You can't misconfigure your way into collecting them, because there's nowhere to put them.

The rest of the security posture follows the same logic:

The agent is outbound-only and opens no listening ports — it can't become a new attack surface on the endpoints it's installed on.
The Docker socket is mounted read-only. Discovery needs to read container metadata; it never needs to start, stop or exec into anything.
Release artifacts are ed25519-signed and verified by the agent at startup, fail-closed. An agent that runs everywhere with root-adjacent visibility is a supply-chain target, and it should refuse to run unsigned code.
mTLS between agent and control plane, RBAC and an audit log on top.

What you actually see

For each discovered instance: which credentials it holds, which production systems those credentials reach, a numeric risk score, a Critical/High/Medium/Low category, and a plain-language explanation of why it scored that way — generated locally, with no LLM in the loop, because a CISO report needs to be reproducible and defensible rather than eloquent.

That access map is the artifact that turns "there is unmanaged software here" into "this laptop can move money."

Two-minute demo

The compose file brings up the control plane, an intentionally shadow n8n instance, and an agent — so you can watch discovery happen on a system that's deliberately misconfigured:

git clone https://github.com/oleg-vdv/AutoGov.git
cd AutoGov/deploy
docker compose up --build

Open http://localhost:8443, log in with the token dev-admin, and within about 30 seconds the agent finds the instance, builds the access map, and shows risk-scored findings. (Demo uses HTTP and dev tokens — local evaluation only.)

Building from source:

make build   # → bin/agent, bin/controlplane, bin/autogov-sign
make test

Pure Go standard library, no external dependencies — which keeps the agent auditable and makes air-gapped installs boring, in the good way.

Current state

Stage 1 (n8n discovery) is MVP and pilot-ready. Generic automations — Make agents, ad-hoc scripts, a Windows agent — are next, with module contracts and stubs already in place for the later stages: honeypot nodes, self-healing and auto-documentation. The module runtime lets those plug in without touching the core, which the tests enforce.

Apache 2.0: https://github.com/oleg-vdv/AutoGov

If you've dealt with shadow automation in your own organization, I'd like to know how you found it — and whether you tried to shut it down or bring it under management. My assumption is that shutting it down mostly fails, because the four hours a week were real, but I'd be glad to hear counterexamples.

"Your company banned ChatGPT. There's a third option."

oleg-vdv — Tue, 01 Sep 2026 16:29:23 +0000

Most companies handle the "employees are pasting customer data into ChatGPT" problem in one of two ways.

Option one: block it. Firewall rules, an acceptable-use policy, a stern email. What actually happens is that people use their phones, personal laptops, and a browser profile you don't manage. You haven't stopped the leak; you've stopped seeing it.

Option two: allow it and hope. Training, a wiki page about "don't paste PII", and an incident waiting to be discovered by someone else.

There's a third option that I don't think gets built often enough: let the request through, but strip the sensitive parts on the way out and put them back on the way in. The model never sees the customer's name. The employee never notices anything happened.

That's what I built AI-Gate for.

The idea in one example

An internal app sends this prompt:

Customer Yerzhan Nursultanuly, ID 900715300005, disputes a charge of 145,000 KZT...

What actually leaves your network:

Customer [PERSON_1], ID [NATIONAL_ID_1], disputes a charge of [AMOUNT_1]...

The model answers about [PERSON_1]. The gateway substitutes the real values back before the response reaches the application. From the app's perspective it just talked to OpenAI normally. From OpenAI's perspective, it never received a single piece of personal data.

The mapping between [PERSON_1] and the real name lives in memory only, encrypted, with a TTL and guaranteed destruction once the response is detokenized. It is never written to disk and never appears in the audit log — the log records types and counts, not values.

Integrating it is one line

For any application already using an OpenAI-compatible client, you change the base URL and nothing else:

from openai import OpenAI

client = OpenAI(
    base_url="http://aigate.internal:8080/v1",
    api_key="<channel key>",
)
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Customer Yerzhan Nursultanuly, ID 900715300005..."}],
)

A detail that matters more than it looks: the real provider API key lives only on the gateway. Applications get a channel key instead. Revoking one team's access stops being a redeployment and starts being a config change — and a leaked application key no longer means a leaked OpenAI key.

Three channels, because the leak isn't only in your code

An egress proxy covers your own applications, RAG pipelines and n8n workflows. It does not cover the two places people actually leak data:

The browser. A Chrome extension intercepts text before it's sent to chat.openai.com, claude.ai or gemini.google.com, masks it through the gateway, and substitutes values back into the assistant's reply as it renders. Deployable via GPO/MDM, so it's not opt-in.

The clipboard. A workstation agent catches sensitive data on its way into local LLM apps — Cursor, Claude Desktop — and detokenizes copied responses. This is the gap neither the browser nor the network perimeter can see, and it's the one that grows every month as AI moves into desktop tools.

Design decisions worth arguing about

Fail-closed, always. Any error in detection or parsing blocks the request instead of passing it through. This will occasionally annoy someone. The alternative is that a parser bug becomes a data breach, and the asymmetry there isn't close: a false block costs a retry, a false pass costs a regulatory fine and a disclosure letter.

Zero dependencies. The core is pure Python 3.11+ standard library. No PyPI at install time, which means it drops into air-gapped environments — exactly the kind of environment where this problem is most acute and where "just pip install" is not an available move. It also means the supply-chain attack surface of a tool that sees all your prompts is approximately zero.

Tamper-evident audit. The log is append-only JSONL with a SHA-256 hash chain. You can't quietly delete the record of a blocked request; verification detects it. For a compliance tool this is the difference between "we have logs" and "we have evidence".

Sensitive prompts can be routed to a local model. The provider registry supports OpenAI-compatible endpoints including vLLM, Ollama and LM Studio, plus the Anthropic Messages API. A policy rule can say: anything containing sensitive data goes to the local model only, never to a vendor. Masking and routing solve different halves of the same problem.

Semantic detection on top of pattern detection. Regexes catch IDs, account numbers and names. They do not catch "our Q3 acquisition target is a logistics company in Almaty with a 40% margin" — a leak with no PII in it at all. An LLM-judge runs over the already anonymized text and flags meaning-level leaks, in flag or block mode, fail-closed if the judge is unreachable.

Try it

cp .env.example .env      # set AIGATE_PROVIDER_API_KEY
python -m gateway.main    # gateway on :8080

or on-prem in a container:

docker compose up -d

The control plane console is at http://localhost:8080/admin — policies, RBAC, dashboards and compliance reports.

Where this is honest about its limits

Detection is imperfect and always will be. Names in particular are the hard case in every language, and a masking gateway that claims 100% recall is lying. Fail-closed helps, the local-model routing rule helps more, and semantic detection catches a class regexes structurally cannot — but the right mental model is defense in depth, not a solved problem.

It's also worth saying plainly: this reduces exposure, it doesn't create legal compliance by itself. What it gives you is the technical control and the evidence trail that a compliance program needs underneath it.

Repo: https://github.com/oleg-vdv/AI-Gateway

I'd be interested to hear how others are handling this. Has your organization landed on block, allow, or something in between — and if you've deployed a masking layer, what broke first?

---

From Infrastructure to Open Source: Lessons Learned Building 4 Security & Automation Tools

oleg-vdv — Fri, 24 Jul 2026 00:47:58 +0000

Coming from a strong sysadmin and infrastructure background, I spent years managing servers, networks, and keeping systems alive. Over time, I realized a fundamental truth: the most dangerous system risks are often the ones you don't even have visible inventory for.

That mindset naturally led me into the world of open source. I started building tools to solve real-world problems around API governance, edge safety, data integrity, and automation.

Here is what I’ve been building in public, what each project taught me, and why these areas matter today:

1. Governing LLM & API Traffic: AI-Gateway

As AI applications move to production, controlling model access, enforcing limits, and monitoring traffic becomes critical.

The Project: AI-Gateway — A lightweight proxy layer designed to secure, route, and manage API requests and policies for AI services.
Key Lesson: Security in the AI era isn't just about firewall ports; it's about context-aware policy management and dynamic traffic control.

2. Safety at the Edge: AffectGuard-HRI

Moving machine learning onto edge devices and microcontrollers opens up huge potential for robotics, but it introduces strict real-time safety constraints.

The Project: AffectGuard-HRI — An open-source framework tailored for human-robot interaction, focusing on real-time safety, intent tracking, and affective monitoring.
Key Lesson: Edge AI demands extreme efficiency. You can't rely on cloud latency when dealing with physical robotic hardware—safety loops must run reliably at the hardware level.

3. Verifiable Data & Audit Trails: ProofByte

In modern SecOps, logging isn't enough—you need verifiable proof of data integrity for compliance and auditing.

The Project: ProofByte — A lightweight tool aimed at data validation, cryptographic verification, and maintaining tamper-evident audit trails.
Key Lesson: Building trust in distributed workflows requires cryptographic validation at every step of the pipeline.

4. Modern Workflow Governance: AutoGov

Processes in enterprise environments often break down due to manual bottlenecks and fragmented oversight.

The Project: AutoGov — An automation and workflow governance engine built to streamline standard operating procedures and administrative checks.
Key Lesson: Good automation doesn't replace governance—it bakes governance into the infrastructure so compliance happens by default.

What’s Next?

Transitioning from maintaining infrastructure to building developer tools in public has been an incredible learning experience.

I’d love to hear from the community: What is your biggest pain point right now when securing edge devices or managing AI API endpoints?

Feel free to check out the repos, leave stars if you find them useful, or open issues/PRs! Let’s connect! 🚀

The post-quantum deadlines are closer than your roadmap thinks — so I built a CBOM scanner

oleg-vdv — Fri, 17 Jul 2026 00:55:53 +0000

Three regulatory clocks are already ticking:

US: EO 14412 / CNSA 2.0 requires post-quantum cryptography for new national-security acquisitions from 2027 — and explicitly names the "cryptographic bill of materials" (CBOM).
UK: NCSC wants full cryptographic discovery and a migration plan by 2028.
EU: member states start transitioning from the end of 2026.

Yet when an auditor asks the first, most basic question — "where exactly does your codebase use RSA?" — almost nobody can answer. Not because it's hard conceptually, but because nobody ever had a reason to keep an inventory of their cryptography. Now there's a reason.

What "harvest now, decrypt later" actually means

Even if a cryptographically relevant quantum computer is a decade away, traffic recorded today can be decrypted then. If your secrets have a shelf life of more than a few years (medical records, financial data, government contracts), classical key exchange is already a liability. That's the logic behind the deadlines — they don't wait for the quantum computer to exist.

One command, three artifacts

I wanted the discovery step to cost one command instead of one consulting engagement:

npx @proofbyte/pqc-radar scan . --cbom cbom.json --report report.md --sarif findings.sarif

This walks the repository and flags quantum-vulnerable cryptography — RSA, ECC/ECDH, DH, DSA, Curve25519, legacy hashes, weak TLS configs — across Java, Python, JavaScript/TypeScript, Go, C#, Rust, Ruby, PHP, C/C++ and nginx/Apache configs. Out come three artifacts:

cbom.json — a CycloneDX 1.6 CBOM: the machine-readable inventory the mandates ask for;
report.md — a human-readable report: every finding with file:line and its ML-KEM (FIPS 203) / ML-DSA (FIPS 204) migration target;
findings.sarif — SARIF 2.1.0, so findings land straight in GitHub code scanning.

It's fast enough to live in CI: hashicorp/vault (4,827 files) scans in about 9 seconds; a weekly corpus workflow re-scans five well-known OSS projects to catch regressions.

Checking live endpoints, not just code

Code is half the story — the other half is what your servers actually negotiate:

npx @proofbyte/pqc-radar scan-tls your-api.example.com github.com

One TLS handshake per host (no payload sent) reports the protocol, cipher, key-exchange group and certificate key — and whether the endpoint already speaks hybrid post-quantum key exchange (X25519+ML-KEM-768). One honesty detail I'm proud of: when the local TLS stack can't name the negotiated group (older OpenSSL builds can't name hybrid groups), the result is reported as inconclusive, not as a finding. Unknown ≠ vulnerable.

What the v0 honestly can't do

Detection is line-pattern based. That makes it fast, broad and dependency-free, but:

it will miss dynamically constructed crypto calls;
unusual code can false-positive;
absence of findings is not proof of absence — the report says so explicitly.

A deep AST engine (integrating PQCA CBOMkit, which covers Java and Python) is the next milestone. There's also a fun war story: the very first regression test caught the scanner's regexes matching their own source code — the fix was masking pattern literals with character-class tricks (RSA_generate_ke[y]), and a permanent guard test asserting the repo only ever flags its own fixtures.

Security posture

This is a tool you're supposed to run on code you can't share, so: fully offline (the scan command makes zero network calls), zero runtime dependencies, read-only, symlinks not followed. scan-tls is the single network feature and touches only endpoints you name. It's MIT-licensed — read the source rather than trusting this paragraph.

Try it

npx @proofbyte/pqc-radar scan .

GitHub: https://github.com/oleg-vdv/proofbyte (issues and PRs welcome — especially new language patterns with fixtures)
CI one-liner: uses: oleg-vdv/proofbyte@v1

I'd genuinely like to hear from anyone whose org has already been asked for a CBOM — by whom, and what format they accepted.