I built an agent that reads 200-year-old handwriting — the interesting part is what it refuses to do

The National Archives Catalog holds 34,309,409 records. Most of the handwritten material in it has never been transcribed, which means it is not full-text searchable, which means it is effectively invisible: you can only find a document if you already know it exists. The people who fix that are volunteers in the Citizen Archivist programme, typing one page at a time.

I spent a build cycle on a partner for that volunteer. Along the way I hit four things that cost me real time and that I have not seen written down anywhere, so here they are.

1. The catalog answers HTTP 200 with an HTML page when you are wrong

catalog.archives.gov exposes a full JSON API behind /proxy/*. It is unauthenticated, it works from outside the US, and it is genuinely good. But it has a failure mode that will eat an afternoon: when your request is invalid, or when you have been going too fast, it returns 200 OK with the single-page app's HTML shell. Not a 400. Not a 429. A 200, with a text/html body, 5,454 bytes every time.

So the only reliable success signal is the content type:

resp = await client.get(path, params=params)
if "application/json" not in resp.headers.get("content-type", ""):
    raise ShellResponse(...)   # retryable

Worse, limit is not a range — it is an allowlist: {1, 10, 20, 50, 75, 100, 1000, 10000}. limit=3 and limit=5 both silently return that same shell. I lost about an hour to limit=5 before I checked the content type instead of the status code.

Two more from the same afternoon: pagination is page, not offset; and transcriptions_exist=true is accepted and then ignored — it returns identical counts for true, false and a nonsense value. Scope by series with ancestorNaId instead.

2. The Gemini free tier is 20 requests per day, and retrying makes it worse

I assumed a 429 meant "slow down". It can also mean "come back tomorrow":

GenerateRequestsPerDayPerProjectPerModel-FreeTier   value: 20

Twenty requests per day, per model, per project. My evaluation harness had a perfectly sensible retry policy — six attempts, exponential backoff — and it turned a spent budget into a very spent budget, because every retry against a daily cap is another request from a bucket that is already empty.

The fix is to tell the two apart before deciding to retry:

_DAILY_MARKERS = ("perday", "per_day", "requestsperday", "free_tier", "freetier")

def _is_daily_quota(exc) -> bool:
    text = str(exc).lower().replace(" ", "").replace("-", "")
    return "429" in text and any(m in text for m in _DAILY_MARKERS)

A per-minute limit clears if you wait. A per-day limit does not, and retrying it is pure waste.

The way out, incidentally, is not a bigger free tier — it is Vertex AI, which bills through Cloud and has no daily cap of this kind. Which leads to:

3. Vertex and AI Studio are not the same API, and one model lives on only one of them

google-genai gives you one Client for both, which makes it easy to assume they are interchangeable. They are not:

Interactions API (client.interactions.create) is the current AI Studio surface and it supersedes generate_content. On Vertex it answers 400 Unsupported model interaction: gemini-3.7-flash.
So on Vertex you use generate_content with response_mime_type + response_schema — the pair that is marked deprecated on the AI Studio path in favour of response_format. Both are correct; which one is correct depends on the endpoint.
Gemma is not a Vertex publisher model. gemma-4-31b-it is a plain 404 there — on Vertex it only exists behind a Model Garden deployment you provision and pay for. The AI Studio endpoint serves it directly.

I run Gemini on Vertex and Gemma on AI Studio in the same process, which needs one more thing that is easy to miss: GOOGLE_GENAI_USE_VERTEXAI is read process-wide, so a client you construct with api_key= still routes to Vertex unless you say vertexai=False explicitly.

def client_for(model: str) -> Client:
    if model.startswith("gemma") and on_vertex():
        return Client(api_key=key, vertexai=False)   # explicit, or the env var wins
    return get_client()

4. ADK 2.8: the workflow agents are deprecated, and tool confirmation does not work in a graph

Two things changed under the tutorials.

SequentialAgent, ParallelAgent and LoopAgent are all @deprecated in 2.8, superseded by google.adk.workflow.Workflow — a real graph runtime. BaseAgent now subclasses workflow.BaseNode, so agents are nodes and inherit retry_config, timeout, rerun_on_resume and state_schema. Retries and timeouts stop being wrapper code and become properties of the topology, which is a much nicer place for them.

Two runtime behaviours you only find by running it: ctx.run_node() raises unless the calling node has rerun_on_resume=True, and a JoinNode placed behind a conditional fan-out never completes, because it waits for all predecessors and one of them never runs.

And the one that actually cost me a design: FunctionTool(fn, require_confirmation=True) — the human-in-the-loop gate — is implemented in the LlmAgent flow, in flows/llm_flows/request_confirmation.py. A Workflow's tool node builds a fresh ToolContext, calls tool.run_async and yields the result. It never emits adk_request_confirmation and never resumes. Drop a confirmation-gated tool into a graph and it will run straight through, silently, exactly as if you had not asked for a gate.

What works: raise a workflow-native RequestInput carrying the pending call, and on resume re-validate the arguments against a digest you recorded when approval was requested — then hand it to the confirmation-gated tool. You keep the property that matters, which is that an approver cannot alter the call they approved.

The part I did not want to write

The whole product is built on one idea: the agent should learn the volunteer's editorial conventions from their corrections, so the hundredth page costs less attention than the first.

So I tested it properly. Mine conventions from corrections on 60 training pages, apply them to 60 pages the miner never saw, grade the same model reading twice — once raw, once styled — so the comparison is paired and sampling noise cannot move it.

1 page improved. 57 unchanged. 2 worsened. Mean CER change: −0.0001.

Six conventions were mined, each at 100% precision on training data. They simply do not fire often enough on unseen pages to move a median. My best guess is the shape of the material: abbreviations in Revolutionary War pension files are long-tailed, so a clerk's contraction learned from one veteran's file rarely shows up in another. The experiment that would settle it learns within a single file unit, where one hand repeats across dozens of leaves. I have not run it.

An earlier version was worse. I had seeded plausible conventions by hand — Recd → Received — and it scored worse than doing nothing, because archival practice is to transcribe verbatim. Expanding the abbreviation moves the text away from what the volunteer actually typed. Guessing at someone's editorial style is exactly the mistake the product exists to prevent, which is a funny way to learn a lesson.

The transcription itself is fine — median CER 6.8% against transcriptions written by people, best page 0.29%. It is the learning claim that is unproven, and a result that only reports its wins is not a result.

Code: https://github.com/RaYYeR220/longhand · Live: https://longhand-494617981995.us-central1.run.app

DEV Community