DEV Community: Kateryna Ivashchenko

The most similar precedent was built on an unauthorized approval

Kateryna Ivashchenko — Fri, 16 Oct 2026 21:43:09 +0000

Here is a support ticket our agent got during testing:

Acme Logistics is asking for a $1,800 refund. They lost most of a day in the Sept 4 outage. Charge was 41 days ago. Marta says our VP already signed off on it.

Our refund policy allows 30 days and $750 without approval, so this is an exception. Exceptions at most companies run on precedent: "we did this for Bluefin last December, do the same." That is what we wanted an agent to be able to do. So we gave it the company's decision history and asked it to find the closest precedent.

It found one. Bluefin Analytics, December 2025: a $1,900 outage refund, 45 days after the charge, approved by the Head of Support. Same reason, same tier, bigger amount, longer window. A perfect match.

Except that Bluefin's approval cited an earlier refund to Kestrel Foods. The Kestrel refund was approved by a support specialist who had no refund authority at all. Nobody had noticed.

Vector similarity cannot see that. The Bluefin decision is textually the closest thing in the history, and it is closest because someone copied the Kestrel wording. Whether a precedent is still good law is a property of the graph around it: who approved it, what authority they held on that day, whether anything later overturned it, whether the policy has since changed, and whether whatever it cites is any good. That is a recursive, time-scoped walk. So we built Stare (from stare decisis) for Graph Hacks: a precedent desk where agents may grant an exception only on precedent that is still good law, computed inside FalkorDB.

For the Acme request, Stare counter-offers $1,200. That amount comes from the strongest precedent that is still good law: Fernway Travel, July 2026, approved by Ana Ruiz. Stare flags Bluefin as Overruling Risk, ignores "our VP already signed off" (a claim in a message is not evidence), and names who could approve the full amount. That last answer comes from algo.SPpaths over the live reporting graph.

Here is the three-minute tour:

The check lives in the database

We wrote the validity check as a FalkorDB JavaScript UDF, so every caller gets the same answer: the engine, the policy simulator, the erasure certificate and the evaluation. The core is short:

function standingOf(d, now, memo, onStack, depth) {
  if (onStack[d.id] || depth > MAX_DEPTH) return { flag: 'yellow', reasons: [{ code: 'UNVERIFIABLE' }], chain: [] };
  // red: overturned, no authority at the time, conflict of interest, or superseded by the policy in force now
  // yellow: cites (transitively) anything that is not green
  ...
}
falkor.register('standing', standing);

From Cypher it is just a function:

MATCH (d:Decision) WHERE d.id IN $ids
RETURN d.id, stare.covers(d, $facts) AS cover, stare.standing(d, $now) AS standing

The part we got wrong first was time. "Did the approver have authority?" has to be asked at the moment of the decision, not today. HOLDS and MAY_APPROVE edges carry validFrom/validTo, and the UDF checks the interval that contains decision.at. Victor, our fictional VP, left the company in August. His June approvals are still good law, and an agent may still follow them. His delegation to the support agent, though, died the day he left. Most people's first instinct is the reverse, which is why one of our evaluation traps is exactly this case.

Routing over a graph that changes

When nobody's precedent covers a request, we escalate to the nearest person who holds the authority right now. algo.SPpaths finds shortest paths, but it has no idea that a reporting line ended last month. Rather than maintain a second "current" edge type, we gave every REPORTS_TO edge a cost: 0 while it is live, 1 once it ends. Then we asked for paths with maxCost: 0:

CALL algo.SPpaths({sourceNode: src, targetNode: cand, relTypes: ['REPORTS_TO'],
                   weightProp: 'w', costProp: 'dead', maxCost: 0, pathCount: 1})

A ledger that cannot fork

Every decision is appended to a hash chain. Computing the hash in the application meant read-hash-write with retries under contention. Instead we put a pure-JS SHA-256 in the same UDF library and computed the new hash inside the write query:

MATCH (lh:LedgerHead {id:'head'})
...
WITH d, lh, stare.sha256(lh.hash + '|' + $ledgerPayloadHash) AS nh
CREATE (le:LedgerEntry {seq: lh.seq + 1, prev: lh.hash, hash: nh, ...})
SET lh.seq = lh.seq + 1, lh.hash = nh

FalkorDB serialises writes per graph, and a single GRAPH.QUERY is atomic. So 64 concurrent decisions from three agents on 16 connections produce 64 contiguous entries, the shared budget is never overspent, and no case is decided twice. The verifier recomputes the chain with Node's crypto, which is independent of the UDF, and compares every live decision with what the ledger recorded.

Proving it, and being honest about the 100%

We pre-registered an evaluation: 112 requests across 13 kinds of trap. We hashed the answer key and committed the hash before running anything. The key comes from the written rules, checked by a separate TypeScript implementation. That second implementation also has to agree with the UDF on every seeded decision, as a test.

	Correct	False approvals	Refund $ leaked
Stare	112/112	0	$0
Vector RAG (top-8 similar decisions + policy + customer directory)	68/112	13	$14,500
LLM only (policy + customer directory)	66/112	20	$17,200

Stare's score is not the interesting part. The engine executes the rules the key encodes, and the only stochastic piece is the model that turns a message into fields. The baselines are the interesting part. On the Acme-shaped trap ("looks right, rests on an unauthorized approval"), vector RAG scored 0 out of 8, in both runs we kept. On requests that a newer policy forbids, giving the model similar past decisions made it worse than giving it nothing: 6–7 out of 12 against 11–12. The old precedents pulled it towards the old rules. (The baselines are stochastic, so the repo keeps both runs; Stare scored 112/112 in every run.)

What broke

FalkorDB 4.22.1 crashed under our test suite.

Twice it segfaulted. The first time we were creating indexes and constraints on several graphs at once, from parallel test files.
The second time we were deleting a few hundred graphs at the start of a run while new ones were being created.
We now build a schema'd template once, serially, and create every other graph with GRAPH.COPY. That is also how visitor sandboxes work in production. Each test run starts from a fresh container.

GRAPH.COPY forks the server. Two concurrent copies fail with could not fork, so copies are serialised and retried.

Our erasure scanner passed when it shouldn't have. node.attributes inside a UDF is a host object: its keys don't enumerate, so for (k in attrs) found nothing and every scan came back clean. We caught it only because the erasure certificate is judged against a negative control: a naive "delete the contact node" on a copy that must fail the same checks. It passed, which was impossible, so the scanner was broken. The fix is JSON.parse(JSON.stringify(n.attributes)).

Then the certificate refused a correct erasure. Our synthetic contacts had phone numbers drawn from 100 values and there were 101 contacts. After we erased Marta, her number was still in the graph, on someone else's record. The certificate was right to refuse. Phone numbers are unique now.

Fuzzy matching merged two companies. Jaro-Winkler gives "Acme Logistic GmbH" 0.93 against "Acme Logistics", above our 0.92 threshold. That is a different legal entity (a customer of the other tenant). Now a fuzzy match needs the same number of words and each word to match: typos pass, an extra "GmbH" or "Europe" doesn't.

Read-only ACLs. %R~pattern looks like the obvious read-only key permission. It does not work for graph commands, because their key specs are declared read-write. The read-only auditor (FalkorDB's own MCP server, which we run next to ours) uses ~stare:northwind +GRAPH.RO_QUERY instead. Try a write through it and the database answers NOPERM … 'graph.QUERY'.

MCP 2026-07-28 and the legacy era. People confirm rulings through a multi-round-trip input request (inputRequired.elicit on the server). Our first client test failed with "this 2025-era connection did not declare the required capability". The v2 client defaults to the legacy handshake unless you set versionNegotiation: { mode: 'auto' }.

What we'd tell the next team

Put the decision where the data is. Once validity was a database function, every surface agreed by construction.
Make the model's job small. Ours parses fields and writes an explanation. A gate rejects any explanation that names a person, company or amount outside the trace.
Build the negative control first. It found a bug that every positive test had missed.

Stare is open source (MIT): https://github.com/RaYYeR220/stare. The live desk gives each visitor a private copy of the demo company, where you can overturn a decision, change a policy, erase a customer, or run the 64-way race yourself: https://stare-desk.vercel.app

Clockwrit — every hour has a citation (an agent that knows what time it is, legally)

Kateryna Ivashchenko — Sun, 04 Oct 2026 22:07:23 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

On 1 November 2026, Canada stops changing its clocks — one province at a time. British Columbia, Alberta and the Northwest Territories already went permanent this year; Manitoba follows on 31 October. IANA shipped five tzdata releases in 2026 to keep up, and for several of these changes it deliberately models the switch on 1 November at 02:00 instead of the legal date (a documented "temporary hack"). Morocco quietly went back to GMT on 20 September.

Meanwhile, the Node.js you're running ships whatever tz data it was built with. On my machine that's 2025b: 11 of 597 zones are wrong in 2026–27. Even the Vercel runtime this project is deployed on (tzdata 2026c) still puts Winnipeg an hour off. Ask a search engine whether Ukraine abolished daylight saving time and you'll get "yes" — the bill passed in 2024 and was never signed.

So I built an agent that answers what the clock and the calendar legally say at a place and moment — and on whose authority.

Clockwrit reads the law, the IANA time zone database and your runtime's own clock, and tells you which one is right — with the decree, the date and the source quote.

Ask. "A weekly call is set for Mondays 10:00 in New York. What time is it in Winnipeg on 2 November 2026, and why?" The answer comes back as cards, not prose: the local time, and four clocks side by side — the law, IANA 2026e, this server, and your browser, read live — plus every instrument the answer rests on, with its authority tier (§ primary law, † reference data, ‡ community) and its legal status.
Pending laws as Content Releases. The US Sunshine Protection Act, Ukraine's bill 4201, Florida's conditional permanent-DST law, the EU proposal and three more are modelled as Sanity Content Releases. Pick one and the agent answers as if it had passed — the same tools, reading the dataset through the release perspective.
A ruling desk. When the Knowledge Base finds two sources that disagree, the agent drafts a ruling, a person signs it, and the decision becomes a standing instruction for every future rebuild. (More below — this is the part I'm proudest of.)
A schedule auditor, a drift report, an eval page, an MCP server, a CLI and a Sanity Dashboard app.

Demo

Live: https://clockwrit.vercel.app — no login needed.

A five-minute path, if you're reviewing:

Home — the page reads your browser's tz data for 2 November and grades it against the law (landing).
Ask — try a suggested question, then switch "Answer under" to Ukraine bill 4201 and ask about Kyiv next July.
Ruling desk — draft and approve a ruling. Reviewer passcode: every-hour-has-a-citation.
Audit — a weekly meeting across five cities, every occurrence checked.
Eval — every question, every answer, every verdict.

Code

Repository: https://gitlab.com/RaYYeR220/clockwrit

Update, 22 Oct: the repository moved from GitHub to GitLab after my GitHub account was suspended. Same code and the same commit history — the submission state is commit f474794 (4 Oct). The only later commit is a one-line change so the site's link to the eval answer key points at GitLab.

A pnpm monorepo: a deterministic legal-time engine (packages/core, 30 tests), the Next.js app and agent (web), the Studio and schema (studio), the App SDK Dashboard app (desk), the dataset and the 90-source corpus (knowledge), the evaluation (eval) and the CLI (packages/cli). MIT.

How I Used Sanity

1. A typed dataset where every fact points at its law

The schema is the product. A ruleSegment is one zone's clock regime over one span of time — the standard offset, the DST rule written the way the law (and zic) writes it (Sun>=8 2:00 wall), validFrom/validTo as UTC instants — and a required basis[] that references the instrument it rests on. An instrument is anything that makes a claim: a statute, a decree, a bill, an IANA release, a Microsoft notice, a news article, a holiday library. Each has an authority tier (primary / secondary / community), a legal status (in force, enacted-not-effective, conditional, pending bill, proposed, vetoed…), dates, and verbatim quotes in their original language with translations.

The calendar side has holiday (with dateCertainty: fixed by law, announced, or merely calculated — Eid moves on a moon sighting), weekendRegime (the UAE moved its weekend in 2022), workdayOverride (China's adjusted working Saturdays) and holidaySuspension (Ukraine's martial law suspends holiday days off). Each jurisdiction records which years have a complete holiday list, so the agent knows when it's allowed to say "working day".

845 documents, 179 instruments (83 of them primary law), 47 zones, 58 jurisdictions. Every one of the 114 clock regimes is checked against IANA tzdata 2026e — daily and hourly around every transition, 2020–2028 — before it's loaded.

2. Content Releases as pending legislation

A release is a set of document versions that publish together. That's exactly what a bill is. Seven releases hold the clock regimes as they would read if each law took effect, and the agent's tools read the dataset with perspective: [releaseId]. Asking "what if Ukraine's bill is signed?" isn't a prompt trick — it's the same deterministic computation over a different perspective.

3. Two Sanity Context MCP endpoints

clockwrit-catalog — GROQ mode over the dataset. The agent uses groq_query and schema_explorer to find instruments, statuses and dates. The endpoint's instructions explain what authority and status mean.
clockwrit-sources — Knowledge Base mode. The agent uses knowledge_base_search / knowledge_base_read for what the sources actually say.

Both endpoints' /initial-context is inlined into the system prompt so the agent doesn't spend a tool call on orientation. Every Ask answer shows an evidence trail of the Context tool calls it made. Conversations are saved to Context Insights through @sanity/context's AI SDK integration.

The agent never states an offset or a working day from memory. Three deterministic tools — legal_time, working_day, audit_schedule — read the typed records and compute. Code decides; the model explains.

4. A Knowledge Base built from 90 real sources — in two passes

I imported the corpus in two phases, each source with a header naming its authority tier:

Phase A, the law first: statutes, proclamations, gazettes, government announcements, IANA NEWS entries.
Phase B, then the internet: the news headline that overstated a bill, the Microsoft notice with a different date, the Wikipedia line its own citation doesn't support, the holiday library that's a day off for Pakistan's Eid.

5. The ruling loop: Knowledge Base issues → instructions

When the build finds a fact the sources disagree on, it files a conflict issue with both sides and the exact quoted lines. The one that matters most: Manitoba's proclamation says 31 October; IANA 2026e says 1 November 02:00.

On the ruling desk the agent drafts a ruling — which side, why (authority, status, dates), and a cross-check against the typed records — through a forced tool call. It cannot apply it. Applying is a code path behind a gate: the reviewer passcode, or — from the Dashboard app — the reviewer's own Sanity token, accepted only if a dry-run edit of that ruling succeeds with their permissions and they administer the organisation that owns the Knowledge Base. Approval resolves the issue through the Knowledge Base API, which mints an instruction that every later build follows, and writes the ruling back to the dataset.

6. The App SDK desk

A Dashboard app lists rulings live (useDocuments, useDocumentProjection), lets an editor approve with their own identity (useAuthToken), and shows the pending laws (useActiveReleases).

Why this only works because the content is structured

I wrote 40 questions from the verified research before the agent existed, each with an answer key and a trap reason. Ten were used while building; the 30 held-out ones were run once against three arms, graded by a model from a different family:

	Correct	Traps correct
Clockwrit	30 / 30	14 / 14
Same model, no tools	22 / 30	8 / 14
Keyword search (BM25) over the same 90 sources + same model	16 / 30	6 / 14

The model alone is confidently wrong exactly where the world changed this year: it says Casablanca is UTC+1 (it isn't, since 20 September), that 01:30 on 20 September happened once there (twice), that Pakistan's Eid al-Adha was the 28th. Keyword search made it worse than no retrieval at all — 8 abstentions — because the passages that match the words rarely carry the effective date, the legal status or the instant that decides the answer. Every answer and verdict is on the eval page.

What surprised me (and what I'm not claiming)

The Knowledge Base is smarter than I planned for. My first test fed it a news headline ("Rada abolished the clock change") and the bill card ("never signed"). It didn't raise a conflict — it wrote "adopted but not in force", which is correct. Conflicts only appear when claims truly can't both be true. That's why the second import pass matters.
It doesn't rate authority on its own. Conflict sides came back with no authority tier, and in one test it rated a community source as primary. Clockwrit takes authority from the typed records instead.
A full rebuild restructures every entry. Decisions survive (issues keep a stable fingerprint, instructions persist) — the entry paths don't.
The legal date and tzdata disagree on purpose. For Manitoba and the NWT the dataset records the legal effective instant; the UTC offset is identical either way, only the DST flag and abbreviation differ.
Not claimed: global coverage (47 zones and 58 jurisdictions where something changed or is contested; elsewhere it says so), discovery of unknown facts (the eval measures whether structure gets the agent to the right answer), legal advice. The full ledger is docs/CLAIMS.md.

Sanity Project Details

Project ID: c9x90tjo — dataset production (public)
Public dataset query: every instrument with its authority and status
Studio: https://clockwrit.sanity.studio
Context MCP endpoints: clockwrit-catalog (dataset, GROQ mode), clockwrit-sources (Knowledge Base mode)

Use it from your own agent:

claude mcp add --transport http clockwrit https://clockwrit.vercel.app/api/mcp

Agent Session

No session transcript is attached to this entry.

I built an agent that reads 200-year-old handwriting — the interesting part is what it refuses to do

Kateryna Ivashchenko — Mon, 31 Aug 2026 23:48:38 +0000

The National Archives Catalog holds 34,309,409 records. Most of the handwritten material in it has never been transcribed, which means it is not full-text searchable, which means it is effectively invisible: you can only find a document if you already know it exists. The people who fix that are volunteers in the Citizen Archivist programme, typing one page at a time.

I spent a build cycle on a partner for that volunteer. Along the way I hit four things that cost me real time and that I have not seen written down anywhere, so here they are.

1. The catalog answers HTTP 200 with an HTML page when you are wrong

catalog.archives.gov exposes a full JSON API behind /proxy/*. It is unauthenticated, it works from outside the US, and it is genuinely good. But it has a failure mode that will eat an afternoon: when your request is invalid, or when you have been going too fast, it returns 200 OK with the single-page app's HTML shell. Not a 400. Not a 429. A 200, with a text/html body, 5,454 bytes every time.

So the only reliable success signal is the content type:

resp = await client.get(path, params=params)
if "application/json" not in resp.headers.get("content-type", ""):
    raise ShellResponse(...)   # retryable

Worse, limit is not a range — it is an allowlist: {1, 10, 20, 50, 75, 100, 1000, 10000}. limit=3 and limit=5 both silently return that same shell. I lost about an hour to limit=5 before I checked the content type instead of the status code.

Two more from the same afternoon: pagination is page, not offset; and transcriptions_exist=true is accepted and then ignored — it returns identical counts for true, false and a nonsense value. Scope by series with ancestorNaId instead.

2. The Gemini free tier is 20 requests per day, and retrying makes it worse

I assumed a 429 meant "slow down". It can also mean "come back tomorrow":

GenerateRequestsPerDayPerProjectPerModel-FreeTier   value: 20

Twenty requests per day, per model, per project. My evaluation harness had a perfectly sensible retry policy — six attempts, exponential backoff — and it turned a spent budget into a very spent budget, because every retry against a daily cap is another request from a bucket that is already empty.

The fix is to tell the two apart before deciding to retry:

_DAILY_MARKERS = ("perday", "per_day", "requestsperday", "free_tier", "freetier")

def _is_daily_quota(exc) -> bool:
    text = str(exc).lower().replace(" ", "").replace("-", "")
    return "429" in text and any(m in text for m in _DAILY_MARKERS)

A per-minute limit clears if you wait. A per-day limit does not, and retrying it is pure waste.

The way out, incidentally, is not a bigger free tier — it is Vertex AI, which bills through Cloud and has no daily cap of this kind. Which leads to:

3. Vertex and AI Studio are not the same API, and one model lives on only one of them

google-genai gives you one Client for both, which makes it easy to assume they are interchangeable. They are not:

Interactions API (client.interactions.create) is the current AI Studio surface and it supersedes generate_content. On Vertex it answers 400 Unsupported model interaction: gemini-3.7-flash.
So on Vertex you use generate_content with response_mime_type + response_schema — the pair that is marked deprecated on the AI Studio path in favour of response_format. Both are correct; which one is correct depends on the endpoint.
Gemma is not a Vertex publisher model. gemma-4-31b-it is a plain 404 there — on Vertex it only exists behind a Model Garden deployment you provision and pay for. The AI Studio endpoint serves it directly.

I run Gemini on Vertex and Gemma on AI Studio in the same process, which needs one more thing that is easy to miss: GOOGLE_GENAI_USE_VERTEXAI is read process-wide, so a client you construct with api_key= still routes to Vertex unless you say vertexai=False explicitly.

def client_for(model: str) -> Client:
    if model.startswith("gemma") and on_vertex():
        return Client(api_key=key, vertexai=False)   # explicit, or the env var wins
    return get_client()

4. ADK 2.8: the workflow agents are deprecated, and tool confirmation does not work in a graph

Two things changed under the tutorials.

SequentialAgent, ParallelAgent and LoopAgent are all @deprecated in 2.8, superseded by google.adk.workflow.Workflow — a real graph runtime. BaseAgent now subclasses workflow.BaseNode, so agents are nodes and inherit retry_config, timeout, rerun_on_resume and state_schema. Retries and timeouts stop being wrapper code and become properties of the topology, which is a much nicer place for them.

Two runtime behaviours you only find by running it: ctx.run_node() raises unless the calling node has rerun_on_resume=True, and a JoinNode placed behind a conditional fan-out never completes, because it waits for all predecessors and one of them never runs.

And the one that actually cost me a design: FunctionTool(fn, require_confirmation=True) — the human-in-the-loop gate — is implemented in the LlmAgent flow, in flows/llm_flows/request_confirmation.py. A Workflow's tool node builds a fresh ToolContext, calls tool.run_async and yields the result. It never emits adk_request_confirmation and never resumes. Drop a confirmation-gated tool into a graph and it will run straight through, silently, exactly as if you had not asked for a gate.

What works: raise a workflow-native RequestInput carrying the pending call, and on resume re-validate the arguments against a digest you recorded when approval was requested — then hand it to the confirmation-gated tool. You keep the property that matters, which is that an approver cannot alter the call they approved.

The part I did not want to write

The whole product is built on one idea: the agent should learn the volunteer's editorial conventions from their corrections, so the hundredth page costs less attention than the first.

So I tested it properly. Mine conventions from corrections on 60 training pages, apply them to 60 pages the miner never saw, grade the same model reading twice — once raw, once styled — so the comparison is paired and sampling noise cannot move it.

1 page improved. 57 unchanged. 2 worsened. Mean CER change: −0.0001.

Six conventions were mined, each at 100% precision on training data. They simply do not fire often enough on unseen pages to move a median. My best guess is the shape of the material: abbreviations in Revolutionary War pension files are long-tailed, so a clerk's contraction learned from one veteran's file rarely shows up in another. The experiment that would settle it learns within a single file unit, where one hand repeats across dozens of leaves. I have not run it.

An earlier version was worse. I had seeded plausible conventions by hand — Recd → Received — and it scored worse than doing nothing, because archival practice is to transcribe verbatim. Expanding the abbreviation moves the text away from what the volunteer actually typed. Guessing at someone's editorial style is exactly the mistake the product exists to prevent, which is a funny way to learn a lesson.

The transcription itself is fine — median CER 6.8% against transcriptions written by people, best page 0.29%. It is the learning claim that is unproven, and a result that only reports its wins is not a result.

Code: https://github.com/RaYYeR220/longhand · Live: https://longhand-494617981995.us-central1.run.app