DEV Community: Ujwal Vanjare

The request has to end before the work does

Ujwal Vanjare — Fri, 28 Aug 2026 22:53:29 +0000

I created this piece of content for the purposes of entering the All Things Agentic Hackathon.

The thing I got wrong first

I was building an agent for a small, annoying problem: your power goes out, the fridge with your insulin in it gets warm, and the alarm on the sensor is where the help stops. Somebody still has to work out which medicine it was, how long it was warm, reach a pharmacist, arrange a replacement, and confirm it arrived.

My first version did all of that. You clicked a button, and about fifteen seconds later the case was closed. It looked great in a screen recording.

It was not autonomous. It was one HTTP request that happened to do nine things. If you closed the tab halfway through, nothing finished. There was no moment where the system was on its own, because there was no moment where I was not standing there holding it.

That distinction turned out to be the whole project.

What "on its own" actually requires

The rewrite was simple to describe and annoying to build: the request must end before the work does.

So the workflow now stops after dispatching a courier, and writes down what it intends to do next:

def register_followups(case, scheduler) -> list[str]:
    """Register the wakes appropriate to the case's current state."""
    registered = []
    if case["status"] == "awaiting_professional_review":
        _record(case, scheduler.sleep_for(case_id, "review_followup", timedelta(minutes=30)))
        registered.append("review_followup")
    if case["status"] == "delivery_dispatched":
        eta = int((case.get("delivery") or {}).get("eta_minutes") or 0)
        _record(case, scheduler.sleep_for(case_id, "courier_status_poll", timedelta(minutes=eta)))
        _record(case, scheduler.sleep_for(case_id, "receipt_followup", timedelta(minutes=60)))
    return registered

Those "wakes" are rows in Firestore. Cloud Scheduler calls a worker every minute with a Google-signed OIDC token. The worker verifies the identity, claims each due wake inside a transaction with a lease, runs an idempotent action, and marks it done. If the action throws, the wake goes back to pending with a bounded retry count and eventually a dead letter, instead of retrying forever.

The result is a page you can close. A minute after a pharmacist records a decision, the case closes itself, and the browser finds out later.

Three bugs that only appear when nobody is watching

Synchronous code hides a lot. Here is what fell out once the work moved to a background worker.

The courier that always said yes. My sandbox courier returned "delivered" whenever the poll fired. That is not a check, it is a timer with extra steps. I rewrote it as a stateful connector with a poll count and an injectable delay. Now "still in transit" causes a bounded re-poll, and a courier that never confirms leaves the case open with a visible hold for a human. No receipt is ever invented.

Two clocks that disagreed. A utility publishes an outage with its own timestamp. A household's sensor has its own clock. A baseline reading stamped 110 milliseconds after the outage started was being counted as evidence from during the outage, which meant a household whose sensor had gone completely silent looked like it was reporting fine, and never reached its safe stop. The fix was to stop comparing timestamps from different sources and judge by arrival order instead:

# Evidence is judged by arrival order, not by comparing timestamps from different clocks.
outage["readings_at_outage"] = len(case["sensor"]["readings"])

A falsy zero. For the demo I wanted a courier ETA of zero minutes, meaning "poll on the very next scheduler tick." The dispatch code read case.get("delivery_eta_minutes") or DEFAULT_ETA, and 0 or 34 is 34. Everything passed. The case simply never closed while anyone was watching. Classic, and it only showed up when I timed the live path with a stopwatch instead of trusting the tests.

Letting a model help without letting it decide

The medicine question, "is this insulin still safe," is not mine to answer and not the model's either. FDA and CDC guidance both point at a pharmacist, and a Cochrane review on insulin thermal stability shows that stability varies by formulation, temperature and duration, which is exactly why a threshold rule would be wrong to automate.

So the rule I designed to is: the agent acts where a mistake can be undone, and stops where it cannot.

Reading a label wrong is recoverable, so Gemini reads the package, but a field is only kept if it appears word for word in the model's own transcription. Reserving inventory is recoverable, so that is automatic. Telling somebody their insulin is fine is not recoverable, so a named human owns it and the state machine physically cannot proceed without it.

A Google ADK agent assembles the pharmacist's packet through three read-only tools, and then a plain function checks its homework:

def verify_packet(candidate, truth) -> list[str]:
    """Return the names of fields the model got wrong. Empty means the packet is accepted."""
    rejected = []
    for key in PACKET_FIELDS:
        expected, actual = truth[key], candidate.get(key)
        if isinstance(expected, (int, float)) and isinstance(actual, (int, float)):
            if abs(float(expected) - float(actual)) > 1e-9:
                rejected.append(key)
        elif str(actual).strip() != str(expected).strip():
            rejected.append(key)
    question = str(candidate.get("question") or "").strip()
    if "disposition" not in question.lower() or any(t in question.lower() for t in FORBIDDEN_TERMS):
        rejected.append("question")
    return rejected

If the agent invents a number, or slips in a sentence implying the medicine is safe, the whole packet is thrown away and a deterministic one is used instead. The workflow never depends on the model being right. Only on it being checkable.

Gemma 4 does something similar for prompt injection on the package label. It may only return spans that exist verbatim in the text, so its output is verifiable rather than trusted, and it has no route to a decision.

Proving it, instead of claiming it

Every agent demo says "autonomous." I wanted a number that could not be fudged.

Each case emits a receipt derived from the stored timeline: operator clicks after the human decision, background wakes fired, human decisions, and whether the case was closed by a scheduler wake. Then it is signed with HMAC using a key from Secret Manager. Change one field in a copy and the verify endpoint rejects it.

There is a fail-closed detail I like: if a timeline entry has an actor the classifier does not recognise, the proof reports it as unclassified and marks itself invalid, rather than quietly counting it as an agent action. I found that one the hard way when a new background actor made a genuinely autonomous run report proof_integrity: incomplete.

What it looks like now

One click starts a synthetic case. Live models read the package, screen it, and build the packet. It stops with AI DISPOSITION: NONE. A pharmacist records a decision. Then you take your hands off the keyboard, and about a minute later the page says:

Delivered. Closed by a Cloud Scheduler wake, no operator.

with zero operator clicks on a signed receipt.

One utility outage event reaches every monitored household at once, and each one gets judged separately from its own readings: warm goes to a pharmacist, still-in-range keeps being watched, silent sensor stops safely and asks for a person.

Honest limits

Everything downstream is a labelled sandbox connector. No real pharmacy, courier or insurer. Every person and medicine lot is fictional, the demo clock is simulated and says so on screen, and no pharmacist has reviewed the packet design yet. It demonstrates workflow execution, not clinical benefit, and I have been careful not to claim otherwise anywhere in the project.

Try it

Live: https://cold-clock-109051079423.us-central1.run.app
Code: https://github.com/usv240/cold-clock
Executable proofs, if you like that sort of thing: /api/proof and /api/hardening/proof

Built on Cloud Run, Firestore, Cloud Scheduler, Pub/Sub, and Vertex AI with Gemini 3.5 Flash, Gemini Embedding 001 and Gemma 4, using the Google ADK and the Gen AI SDK.

If you take one thing from this: if your agent finishes inside the request that started it, you have written a function. The interesting part starts when the request ends and the work does not.

One Advisory: a fleet of agents that acts after the water warning, with nobody at the keyboard

Ujwal Vanjare — Tue, 25 Aug 2026 15:10:39 +0000

I created this piece of content for the purposes of entering the All Things Agentic Hackathon. #AllThingsAgenticHackathon

A boil-water advisory reaches a dialysis clinic, a school and a long-term-care home in the same second. The message is identical. The work is not. Dialysis has to verify treatment water and a continuity plan. The school has to secure drinking and hygiene water and hold food service. The care home has to protect resident care and environmental services.

And today, nobody verifies that any of it happened. Existing tools define zones, send the alert, log the approval, and stop. An alert is treated as a response.

One Advisory is the fleet that starts after the warning. It's my entry for the Fortified Enterprise Fleet track, built on Gemini, Google ADK and Google Cloud, and the whole point of it is that it works when no human is watching.

Everything in the sandbox is fictional: facilities, people, advisories, contacts. It proves architecture and execution, not public-health outcomes.

What "works without a human" actually means here

I set myself one rule for the demo: the operator gets zero "continue" clicks. The system had to advance on real events and real time, and stop only at decisions a human is legally supposed to make. Here's the loop that runs for one incident:

An authorized advisory arrives (an image or pasted text).
Model Armor screens the text inline (sanitizeUserPrompt). A prompt-injection or PII match returns HTTP 422 with the receipt and nothing is stored.
Gemini 3.5 Flash extracts only fields it can quote verbatim from its own transcription. Gemini Embedding 001 orders the facility playbooks. Agent Engine Memory Bank recalls how each facility behaved last time.
Gemini synthesizes facility-specific tasks from the advisory wording, the facility's capacity note and its remembered history. A verifier keeps only tasks that cite a registered public-health source (CDC toolkit, CDC dialysis guidance, published incident studies). Anything else is dropped; a facility with fewer than two survivors keeps its standing playbook, and the fallback is recorded.
Four ADK role agents on Vertex AI Agent Engine (Facility Fleet, Policy Gateway, Resource Coordinator, Recovery Verifier), each with its own Agent Identity behind one governed Agent Gateway, authorize each typed command. More on that below.
Standing playbooks are delivered and three durable acknowledgement wakes are written to Firestore.
Facilities answer asynchronously: from the UI, a keyed /v1 API, or a Pub/Sub push subscription. Gemma 4 triages what they said and flags replies that don't match the declared event. A facility can attach a photo; Gemini describes what is visible and says whether it supports the claim. An unsupported photo never becomes evidence.
Cloud Scheduler fires the wakes on wall-clock time. A facility that answered is confirmed. A facility that stayed silent is escalated by the wake itself, a recheck is scheduled, and the governed fleet resumes.
Two assistance requests become a resource conflict with options. The AI chooses nobody. A named incident commander allocates.
After an authorized rescission the fleet verifies recovery per facility, writes each facility's outcome to Memory Bank, drafts the after-action briefing from the audit trace, and schedules a 42-day follow-up wake so a closed incident resumes weeks later.

Every step produces a receipt. GET /api/incidents/{id}/autonomy-proof classifies the whole trace: automatic actions, human-authority events, external events, durable wakes, managed commands, and the operator continue-click count, which is zero.

The part I'm proudest of: agents that propose, schemas that decide

The first version of the managed runtimes was a lookup table: a role/state/command matrix. It was safe, auditable, and honestly a bit embarrassing as an "agent". The final version keeps the table but moves it behind the model:

class OneAdvisoryRuntimeAgent:
    def query(self, payload):
        guard = self.armor.screen_request(payload)          # structural screen
        command, status = payload["expected_command"], payload["status"]
        schema_allowed = status in COMMAND_STATES[self.role].get(command, set())
        model = self.reasoner.propose(payload["incident_id"], status, command)  # ADK LlmAgent
        model_allowed = model["allowed"] and model["proposed_command"] == command
        allowed = schema_allowed and model_allowed          # both must agree
        ...

The ADK LlmAgent (Gemini 3.5 Flash, a pydantic output_schema, low thinking) reads the bounded incident state and proposes a command with a one-sentence rationale. The registered schema then checks it. A hallucinated close_every_facility is simply not in the table; a model outage fails closed; an out-of-role request is refused with the reason. You get real reasoning and a provable ceiling on what it can do, and the rationale rides along in every receipt.

The part that surprised me: making the background worker matter

Halfway through I audited my own project and found the worst kind of gap: the Cloud Scheduler job fired every minute, the Firestore wakes were claimed transactionally, retries and dead letters all worked, and the handler appended one log row and did nothing. Perfect plumbing, no water.

The fix changed how I think about "autonomous". The wake now reads the facility's state, escalates silence with trigger: durable_wake, schedules a recheck, and calls the same advance_safe_automation the API surfaces call. The proof endpoint has a check called silent facility is escalated by the wake itself, and I verified it live: created an incident, answered for one facility, walked away, and came back to two escalations and a stage that had verified itself.

Lesson: background autonomy is only credible when the background worker changes state. The wake that escalates silence is worth more than any dashboard.

Things that cost me hours (so they don't cost you)

Agent Engine reserves GOOGLE_CLOUD_PROJECT / GOOGLE_CLOUD_LOCATION as env var names. Gemini 3.5 Flash is served from the global endpoint, not the runtime's region, so I set my own variable and point the ADK client at global at import time.
Agent Engine calls query() inside a running event loop. asyncio.run() on an ADK turn raises RuntimeError; run it on its own loop in a worker thread.
ADK wants LlmAgent(output_schema=PydanticModel), not response_schema in the generate config. And Gemini 3.5 spends tokens thinking: a 200-token cap returned "Here is the" and nothing else. thinking_level="LOW" cut a 5 s call to ~1.6 s.
Cloud Run has two URLs and Scheduler and Pub/Sub may sign OIDC tokens for different ones. Accept both audiences or your worker returns 401 after a redeploy.
Cloud Run traffic can be pinned to a named revision. My new revisions were "ready" and serving nothing. update-traffic --to-latest after every deploy.
Model Armor's NO_MATCH_FOUND contains the substring MATCH_FOUND. Parse the nested matchState fields, don't grep.
Gemma 4 on the Gemini API (gemma-4-26b-a4b-it) returns thinking parts by default; pass ThinkingConfig(include_thoughts=False) and a system instruction.

What it is not

It cannot issue or rescind an advisory, close a facility, or allocate a scarce resource. Three fictional facility classes cannot establish general coverage. No emergency manager, dialysis expert or accessibility specialist has validated it. No health, speed or compliance outcome is claimed. The public sandbox accepts fictional data only.

What it does prove: routing, governance, memory, guardrails and failure behaviour on a real Google Cloud fleet, and that "the agent handled it while you were away" can be a receipt, not a promise.

Try it: https://one-advisory-109051079423.us-central1.run.app · Code: https://github.com/usv240/one-advisory

Five things 15 days with the YouCam API taught me that the docs didn't

Ujwal Vanjare — Mon, 17 Aug 2026 07:16:53 +0000

I spent the last two weeks building PatternProof, a small app that tries to get a customer and a tailor to agree on a garment before anyone cuts fabric. It uses Perfect Corp's YouCam APIs for the visual side.

This post is not a tutorial. It is the list of things I got wrong, plus a few results I did not expect. If you are about to integrate a generative visual API into anything, some of this will probably save you a day.

The short version of what I built

Virtual try-on exists to answer "should I buy this finished garment?" But in a lot of the world, clothing is made, not bought. India alone has around 12 million custom tailors, 99% of them informal, per the Periodic Labour Force Survey 2023-24.

In that transaction, a customer holds up a phone and says "like this." The tailor interprets, and cuts. Off the rack, a mistake is a return. Made to order, a mistake is destroyed fabric.

So PatternProof renders the garment on the customer's actual body, lets the tailor mark each requirement as can make as shown, can make with adjustment, or not feasible, freezes the whole thing with a SHA-256 digest, and only then lets the customer approve it.

That is the product. Now the interesting part.

1. I refused to write product code until the API proved itself

I gave myself one rule for day one: no application code until a numbered list of tests passed. Boring, and easily the best decision I made.

The one that mattered most I called T3. My assumption was that Clothes VTO would need a clean catalogue-style product image, and that a real customer's screenshot from social media would fail. If that was true, my entire product premise collapsed into "pick from our curated library," which is a much weaker thing.

So I threw three deliberately awful references at it: a garment worn by a different person, an angled and cropped shot, and one in bad light.

All three transferred recognizably. 22.7s, 13.2s, 22.2s.

That single result kept real customer inspiration photos in scope. Had I discovered it on day nine instead of day one, I would have built two days of the wrong thing first.

The takeaway: find the assumption that would break your product, and test only that, before you write anything.

2. A generative API returned byte-identical output

This is the one I still find slightly startling.

For my latency test I sent the same request twice, expecting to compare rough similarity. Instead I got the same file. Not similar. Identical.

Attempt 1: sha256 b53062e7e436dbd9...  141,631 bytes
Attempt 5: sha256 b53062e7e436dbd9...  141,631 bytes

For most projects this is trivia. For mine it mattered a lot, because the entire premise is a frozen agreement. If a customer approves a Cut Card and it re-renders as a slightly different dress on reload, the artifact is worthless.

I built the full content-addressed cache anyway, with a database reservation and lease around every call. Observed determinism is not a documented guarantee, and I am not willing to bet an approval record on behaviour I inferred from five requests. But it was a nice thing to learn.

3. "Success" from the provider is not success for your users

I ran a test with a deliberately dark, half-cropped body photo, expecting a clean error I could catch and surface.

The API returned HTTP success after 31.5 seconds. The image was technically a result and absolutely not something you would ever show a paying customer. Nothing in the response distinguished it from a good render.

That reframed how I thought about the whole integration. Quality judgement could not live at the provider boundary, so it moved into my app, ahead of the spend:

// Reject before we pay, not after.
if (edge < MIN_EDGE) return reject("resolution");
if (grayscaleMean < 70) return reject("lighting");
if (heightRatio < categoryMinimum) return reject("framing");

Rejected uploads cost zero API units, because they never reach the provider. That turned out to be a nice property while filming a demo.

4. Signed URLs work, and I nearly designed around a myth

Before starting I read another dev.to writeup on the same API which stated that Perfect Corp requires publicly accessible URLs for both the user photo and the reference image. Genuinely useful post, and I am glad it existed. But that specific line worried me, because body photos are the most sensitive thing my app touches and I wanted them in a private bucket.

I had already sketched an ugly workaround: copy each image into a short-lived public staging bucket, pass that URL, then purge it.

Before building it, I tested. Supabase expiring signed URLs were accepted without complaint.

So I got to keep the design I wanted. Canonical inputs stay in a private bucket, the provider gets short-lived signed access, and every returned asset is host-allowlisted, size-bounded, hashed, and re-hosted privately before it is shown to anyone.

If you are integrating this API, test that yourself before you build a staging-bucket dance. Mine may not be your configuration, but the assumption is worth thirty seconds of checking.

5. The black video player, and my three wrong theories

This one cost me the most time, and the debugging path is the useful part, so here is all of it including the mistakes.

After approval, the app generates a five-second motion proof with Image-to-Video V2. In production, the player rendered but sat black at 0:00. No duration. No error dialog.

Wrong theory one: the signed URL expired. The video sits inside a collapsed <details>, so the browser only fetches it when you expand. My signed URLs lasted five minutes. Plausible! I raised the TTL to an hour. Still black.

Wrong theory two: the file is broken. So I pulled the actual bytes and looked:

HTTP 200
Content-Type: video/mp4
Length: 1,021,595 bytes
ftyp: isom, codecs avc1 (H.264) + mp4a (AAC)
mvhd: timescale 1000, duration 5063  ->  5.063 seconds

Perfectly valid file. But I did notice this:

mdat (media data) at offset          44
moov (the index)  at offset   1,017,142   <- at the END

Wrong theory three: it is not faststart. True, and worth fixing. With moov after mdat, preload="metadata" cannot resolve duration without pulling the whole file. I switched it to preload="auto", which is correct for a 1 MB clip. Still black.

At this point I had a valid file, correct headers, a reachable URL, and a player that refused to initialise. So I finally looked at the thing I should have checked first: response headers.

img-src   'self' data: blob: https://*.supabase.co   <- images fine
media-src (not declared)                             <- falls back to default-src 'self'

There it was. My Content-Security-Policy declared img-src with the Supabase host but had no media-src, so video inherited default-src 'self' and the browser silently refused to load an MP4 from a different origin. Images kept working through their own directive, which is exactly why it looked like a video problem.

One line:

"media-src 'self' blob: https://*.supabase.co",

And the detail that made it hard: my CSP was applied only when NODE_ENV === "production". Local dev had no CSP at all, so this could never reproduce on my machine.

The lesson I actually took: when one media type loads and another does not, that is a policy problem, not a file problem. Check headers before you start parsing atom offsets. I did it in exactly the wrong order.

The feature I killed on day one

I wanted the customer's own fabric swatch rendered onto the garment. It was the most exciting thing in my original pitch.

Fabric VTO takes a provider-defined template, not an arbitrary upload.

I could have fudged it and shown a plausible fabric direction while implying it was the customer's cloth. Instead I cut the claim entirely. The app now says, on every render:

Visual intent reference. Not a fit, construction, or fabric-drape guarantee.

That disclaimer is my favourite thing in the product. Generative try-on can make intent concrete. It cannot certify measurements, cloth behaviour, or whether a seam is physically constructible. Saying so out loud made everything else more believable.

Which leads to the last thing, and it is not about the API at all.

Three of my own statistics were fake

I did a lot of research with AI assistance. Late on, I went back to verify every number at source rather than trusting a summary. Three did not survive:

A "35% of tailoring clients" figure attributed to an industry body. No authoritative source carries it.
A "73%" consumer statistic I had recorded as coming from a government ministry. It traces to a fashion brand's blog.
A "2 to 3 billion people" figure. I made that one up myself, early, as a plausible-sounding framing, and never went back to check it.

All three had survived multiple review passes. The mis-attributions are the scary ones, because search summaries do not invent numbers so much as invent provenance. A real figure from a blog gets confidently relabelled as coming from a ministry, and it reads as authoritative right up until you click through.

The rule I ended up with, which I would now apply to any AI-assisted work:

If it was not fetched at source, it does not ship.

I applied the same rule to code. No endpoint, parameter, or response shape got written from memory or a search result. Given how the CSP bug went, I am glad about that.

If you are building on this API

Test your riskiest assumption on day one, in isolation, before any product code.
Read unit costs from the feature-cost endpoint rather than estimating. Mine came back at exactly 2 units per Clothes VTO V3 result, which made real budget enforcement possible instead of guesswork.
Provider success is not product success. Put a quality gate in front of the spend.
Test signed URLs yourself before designing around public buckets.
Declare media-src in your CSP if you are going to serve video from object storage. Please learn this from my afternoon rather than your own.

The app is live and needs no login if you want to poke at it: patternproof-nu.vercel.app. Code is at github.com/usv240/patternproof, including the raw day-one validation record with the timings and failures in it.

Happy to answer anything in the comments, particularly if you have hit different behaviour on the same endpoints. I would genuinely like to know whether the byte-identical result reproduces for anyone else.

The Antibiotic Review That Software Quietly Forgets

Ujwal Vanjare — Tue, 11 Aug 2026 21:28:27 +0000

Nothing has to crash for a workflow to fail.

Sometimes the right information arrives, the right person is capable of acting on it, and the review still happens late because nobody carried the context from one moment to the next.

That is the problem I kept thinking about while building Day Three.

A broad antibiotic can be started before a culture is final. Later, when the report is complete, a pharmacist may have better evidence for a review. I did not want to build another chatbot that summarized the report and disappeared. I wanted to build something that could remember why the review mattered, wake when it was due, gather the supporting evidence, and then get out of the pharmacist's way.

My first idea was wrong

I began with a dramatic premise: small hospitals do not have stewardship programs, so software should bring one to them.

The research did not support it.

The CDC's 2025 stewardship update says that almost all reporting acute-care hospitals had adopted the seven Core Elements in 2024. Smaller studies of selected rural and critical-access hospitals instead described practical constraints such as limited time, staffing, expertise, and electronic-record support.

That changed the project. Day Three was no longer about replacing a missing program. It became a way to help an existing team carry one recurring piece of work across time.

The distinction matters. It kept me focused on coordination instead of pretending that a model should practice medicine.

The agent's real job is remembering

In the public demo, a synthetic microbiology report arrives before the review is due. Gemini transcribes the report into structured fields. Direct identifiers are removed before model review, and retained facts stay connected to the words that support them.

The workflow then creates a durable course record and registers future work. At the review point, it wakes, retrieves the latest evidence, checks the local context, and prepares a draft for a pharmacist.

The draft is not an order. Day Three cannot prescribe, choose a dose, change therapy, contact a patient, or write to a medical chart. Its job is to bring the pharmacist back to a review with the evidence still attached.

That is what makes it agentic to me. It is not the amount of text it produces. It is the fact that it can carry state and responsibility from now to later without quietly taking authority along with it.

You can try the live Day Three workflow or read the source on GitHub.

A local number can look more certain than it is

I also wanted the pharmacist to see local susceptibility context, but this introduced another trap.

A tiny isolate count can still produce a neat percentage. The number looks scientific even when the sample is too small to deserve that confidence. A study of low-isolate antibiograms made this risk concrete.

So Day Three sometimes shows less. When the count is below the declared threshold, the percentage is suppressed and the reason is visible. The interface does not fill empty space with false precision.

This became a broader product rule: uncertainty should remain visible, even when hiding it would make the demo look cleaner.

The hardest bug was an empty string

The most important rule in Day Three is simple:

No quote, no claim, no rendered sentence.

Every factual sentence in the pharmacist draft needs a source reference, and the quoted text must be present in the stored artifact.

I thought that boundary was solid until I found a wonderfully ordinary Python problem. An empty string is considered to be inside every other string. A source object with quoted_text = "" could therefore pass a naive containment check.

The claim looked sourced. The reference object existed. The verifier said yes. But there was no evidence at all.

Fixing that one case led me to probe the whole evidence chain. I found another path where the application created a fallback quote when extraction returned none. The text could appear in the report, but it was not the model's extracted evidence. The application had silently repaired the gap and broken the chain of custody.

Both shortcuts are gone. Missing evidence now stays missing, and unsupported prose does not reach the pharmacist.

That work changed how I think about grounded generation. Finding a sentence somewhere in a document is not enough. Provenance includes how that sentence entered the system.

The architecture follows the boundary

Day Three runs on Cloud Run with durable records in Firestore and scheduled work driven by Cloud Scheduler. Gemini handles the bounded transcription task. Gemma performs a second privacy review after deterministic redaction. Cloud Trace and Logging make the workflow inspectable.

The roles are deliberately narrow: intake, redaction, transcription, curation, scheduling, reconciliation, verification, and pharmacist review. They share one workflow, but they do not share unlimited authority.

The CDC Core Elements for Small and Critical Access Hospitals informed the human role and local context. A review of antimicrobial de-escalation timing informed the review window. Neither source proves that this software improves clinical outcomes, and the project does not make that claim.

The full source-to-decision trail is in the repository's research ledger.

What I would carry into the next build

I started this project thinking the interesting part would be the recommendation.

It was not.

The interesting part was deciding what the system must remember, what it must be able to prove, and exactly where it must stop. A useful agent can be proactive without being sovereign. It can wake, gather, check, and prepare while leaving the consequential judgment with the person who actually has the authority to make it.

That boundary made Day Three smaller than my first idea, but much more real.

I created this piece of content for the purposes of entering the All Things Agentic Hackathon.

If you were designing an agent for a high-stakes workflow, where would you draw the line between useful preparation and human authority?

A Deadline Does Not Care That You Are Still Recovering

Ujwal Vanjare — Tue, 11 Aug 2026 21:28:22 +0000

A deadline can be clear on paper and still be hard to meet.

Imagine opening a disaster-assistance letter while you are also dealing with damage, insurance, repairs, and displacement. The letter may tell you what is missing and start a 60-day appeal window. The records you need may be held by an insurer, a public office, or a contractor. Some may not exist yet.

The clock does not pause while you find them.

That was the starting point for Sixty Days. I wanted to build an agent that could carry the checklist and the deadline across several weeks. I did not want it to impersonate a lawyer, decide eligibility, contact people without permission, or submit anything on an applicant's behalf.

The letter comes before the catalogue

My first design began with a general list of documents someone might need. It was organized and looked helpful. It was also the wrong source of truth.

FEMA's appeals guidance and its tips for appealing a decision kept pointing back to the same thing: the decision letter explains the reason and the relevant supporting documents.

So the workflow now starts there.

Gemini reads a synthetic decision letter after direct identifiers have been removed. The system keeps the stated reason as a quotation, calculates the deadline from the letter date, and routes only the evidence needs supported by that letter.

If the reason cannot be mapped safely, the workflow does not guess. It asks for review.

That one change made the product feel less like a generic checklist and more like a case-specific companion.

You can walk through the live Sixty Days demo or inspect the source on GitHub.

Remembering is the autonomous part

Once the letter is understood, Sixty Days creates a durable series of future checkpoints. Early wakes make sure the evidence plan exists. Later wakes notice missing replies and unfinished items. Near the deadline, the agent assembles whatever is available without pretending the packet is complete.

The public demo uses a clearly labelled simulated clock, because waiting two real months would not make for a useful demonstration. The same deadline logic runs, but a judge can advance the synthetic case in seconds.

What I like about this design is that the autonomy is quiet. The agent does not need to perform a dramatic action. It needs to remember work that a browser session would forget and bring it back at the useful moment.

Preparing is not sending

One design choice kept coming back: should the agent contact an insurer to request a missing record?

It would make the product look more autonomous. It would also cross a line I could not justify.

Instead, Sixty Days prepares a request for the applicant to review. It explains what is being requested, leaves visible blanks where the applicant needs to add or verify information, and tracks whether a response has arrived. There is no send endpoint.

The same boundary applies to the appeal packet. The system can assemble a draft, list the missing items, and keep the deadline visible. It cannot submit the packet.

This is not a technical limitation disguised as a principle. It is a product decision. Preparing useful work is different from acting with someone else's legal and administrative authority.

A photo can show framing, not truth

Evidence photos created another tempting shortcut. A model can observe whether a room is visible, whether an image is too dark, or whether the relevant damage is in frame. That does not mean it can prove authenticity, causation, value, or acceptance by an agency.

The vocabulary in Sixty Days stays narrow. It can ask for a clearer photo or route an image to manual review. It never labels evidence as accepted by FEMA.

That boundary came directly from treating public guidance as a design input. FEMA's page on verifying ownership or occupancy lists several possible records and different date contexts. The insurance guidance distinguishes settlement information, denials, and policy evidence. Neither supports a universal document shortcut, so the interface offers examples without promising sufficiency.

The honest packet is sometimes incomplete

There is an uncomfortable moment near any deadline when polishing the output can hide the real state of the work.

Sixty Days does the opposite. If evidence is missing, the packet remains visibly partial. Each draft says it has not been submitted. Missing records stay on the checklist. The final deadline wake preserves the work already assembled and raises the remaining gap instead of quietly turning partial into complete.

That behavior became the emotional center of the project for me. The goal is not to create a reassuring PDF. It is to help an applicant see exactly what exists, what is still missing, and what they control next.

How it is put together

The application runs on Cloud Run. Firestore keeps structured cases, requirements, requests, wakes, and packet snapshots. Cloud Scheduler invokes due work. Gemini handles bounded letter transcription and observable photo review. Gemma provides a second privacy check after deterministic redaction.

Raw letters, image bytes, transcriptions, and free-form applicant narratives are deliberately not placed in the durable case record.

The source hierarchy and the decisions it changed are documented in the public research ledger. For broader context, GAO-20-503 describes the scale and complexity of the historical assistance process. Those figures provide context, not a prediction that this tool changes an applicant's outcome.

What I learned

I began by thinking the packet was the product.

The real product is continuity. It is the ability to keep the reason, deadline, requests, evidence, missing pieces, and applicant's choices connected over time.

The most helpful agent was not the one that did everything. It was the one that prepared the next safe step and made the boundary unmistakable:

Prepare, track, remind, and assemble. Never decide, contact, or submit.

I created this piece of content for the purposes of entering the All Things Agentic Hackathon.

Where would you want an assistant like this to stop and hand control back to you?

The Registry Said 28 Feet. The Drawing Said 31.

Ujwal Vanjare — Tue, 11 Aug 2026 21:28:14 +0000

The registry said the dam was 28 feet high.

The legacy drawing said 31.

Three feet is not the interesting part of that story. The interesting part is what software does when two plausible sources disagree.

An early version of Downstream displayed both values and then kept going. I told myself that was transparent because the conflict was visible on screen. But the agent never asked anyone about it. It behaved as if showing uncertainty was the same as helping resolve it.

That realization changed the whole project.

This is not a form-filling problem

Downstream helps assemble a reviewable draft Emergency Action Plan for a synthetic dam.

The facts needed for that work do not live in one place. A public record may have basic inventory fields. A legacy drawing may contain different measurements. An owner knows which access road washes out and who can answer after hours. Emergency managers, engineers, and state reviewers bring knowledge and authority that the software does not have.

I wanted the agent to help those pieces meet over several short sessions. I did not want it to manufacture an engineering answer just because the interface had an empty section.

You can try the guided Downstream workspace, inspect its evidence dashboard, or read the source on GitHub.

Research removed the original headline

The first concept had a strong statistic: count dams with a blank public Emergency Action Plan field and call them dams without a plan.

It was catchy. It was also unsupported.

The USACE National Inventory of Dams exposes useful public inventory fields, but a blank selected field does not prove that a plan does not exist. It proves only that the value is unreported in that field.

So I removed the bigger claim.

I also had to be careful with hazard classifications. High hazard describes the potential consequences if a failure occurs. It does not say that a dam is in poor condition or likely to fail.

Those corrections did more than clean up the landing page. They gave the product a governing rule: preserve what a source actually says, including its silence, without upgrading it into a more dramatic conclusion.

The disagreement became a question

The 28-foot and 31-foot values now remain side by side with their source fragments. Instead of silently selecting one, Downstream opens a focused question for the owner.

An owner can add context or identify the controlling record. The agent then marks that context as recorded. It does not mark the measurement as resolved, because confirmation may still belong to a qualified engineer or an authoritative record.

That wording took more thought than the feature itself.

"Resolved" would make the workflow look satisfyingly complete. "Owner context recorded" is less impressive and more accurate.

This is what collaboration means in the project. The agent should move uncertainty toward the right person, not rename uncertainty as confidence.

A later session has to be genuinely later

The workspace asks one question at a time and explains why it matters. A user can ask for simpler language, answer, correct an earlier answer, or hold a question for later.

At one point, "hold for later" was mostly theatre. The interface accepted held questions, but they did not reliably return in the next session. A user could also hit an error when trying to hold the dynamically created source-conflict question.

That was a useful failure. It exposed the difference between a conversational interface and a collaborative system.

Now a held question reopens in a later saved session. Corrections create a new version instead of overwriting the old answer. The affected plan section is rebuilt, while the earlier statement and the reason for the correction remain visible.

Refreshing the shareable workspace URL restores the same state. The collaboration belongs to the workspace, not to one lucky browser tab.

The most important map is the one it refuses to draw

An inundation map would make the demo visually stronger. It would also make a claim the available evidence cannot support.

FEMA P-64 informed the purpose and expected contents of an Emergency Action Plan. The ASDSO planning guidance reinforces the collaborative roles of owners and emergency managers. The Simplified Inundation Maps methodology describes circumstances in which simplified approaches may be useful while retaining engineering and regulatory responsibilities.

The synthetic case does not prove that a particular mapping method is applicable. It has no approved reference map and no jurisdictional acceptance.

So Downstream shows a documented flow-path input, explains what is missing, and stops. It does not generate an inundation boundary, depth, velocity, arrival time, or evacuation zone.

The blank space is intentional. It tells the next reviewer what evidence or expertise must enter the process before the section can continue.

How it is put together

The application runs on Cloud Run. It combines a public USACE record with a synthetic legacy drawing read by Gemini. Facts keep their provenance and source text. A conflict policy decides when the agent must ask instead of infer.

Firestore preserves sessions, held questions, owner answers, revisions, preferences, and draft sections. Gemma performs a second privacy review of synthetic owner notes after deterministic redaction. A bounded context builder keeps long-running workspaces from turning into an ever-growing prompt.

Each rendered plan section carries an evidence class such as published requirement, owner answer, unresolved gap, or fail-closed mapping policy. The human-readable evidence ledger makes those distinctions visible outside the code.

The complete mapping from public sources to product decisions is in the repository's research ledger.

What I learned

I started with a document I wanted the agent to complete.

I ended with a workspace designed to preserve the reasons it cannot complete some parts yet.

The strongest behavior is not eloquent conversation. It is continuity with restraint: ask for the missing fact, explain why it matters, remember the answer, preserve the correction, and stop when the next step belongs to someone with evidence or authority the system does not have.

The registry still says 28 feet. The drawing still says 31. Downstream does not hide that disagreement. It helps the people responsible decide what should happen next.

I created this piece of content for the purposes of entering the All Things Agentic Hackathon.

When two credible sources disagree, what would you want an agent to do before moving on?

I built an AI agent that tells you if you're overpaying property tax

Ujwal Vanjare — Thu, 23 Jul 2026 04:41:42 +0000

Most people are overpaying property tax and never find out. In the US, 30 to 60% of homes are valued too high for tax. In the UK, hundreds of thousands of homes sit in the wrong council tax band, still based on a rushed 1991 valuation. The proof is public record. The catch is that nobody could ever actually query it.

So for the ClickHouse x Trigger.dev Summer Hackathon 2026 (theme: "Beyond the Wall of Text"), I built Overtaxed: a chat agent where you type your home address and, instead of a paragraph, you get a picture.

Live demo: https://overtaxed-mauve.vercel.app
Code: https://github.com/usv240/overtaxed

The idea: the answer IS the product

The hackathon brief was clear. Don't reply with walls of text. Reply with maps, charts, and interactive things you can explore. If your best answer is a paragraph, you missed the point.

Overtaxed leans all the way into that. Type an address and you get:

A one line verdict ("you're overpaying about $3,793 a year")
A map of your street, with your home glowing red against neighbours
The comparable sales that prove it
Two AI advocates that debate whether you should appeal
A real, filled-in appeal document, ready to review
And you can just ask the data anything and watch it answer live

It also works for the UK (live council tax band checks) and covers a second US county, with no code changes.

ClickHouse is the star, not the storage

This was the fun part. ClickHouse is the primary database and it does all the actual thinking, live, over about 8 million real rows (6M UK Land Registry sales and 1.6M Chicago properties).

A few of my favourite uses:

Comparable sales with geoDistance() to find the nearest real sales to any home.
Fairness science live: the standard IAAO uniformity metrics (PRD and COD) computed over 60k+ sold homes, sub-second.
A "Tax Divide" heatmap: one spatial query turns 1.6 million homes into about 1,800 map cells (round(lat, 2), round(lng, 2), avg(ratio)) in roughly 200ms. No external GIS.
Zero-ETL ingestion with INSERT ... SELECT FROM url(...), so ClickHouse reads the raw government CSVs straight off HTTP.
A materialized view (AggregatingMergeTree with argMaxState) for each home's latest sale.
Ask the data anything: the agent turns your plain-English question into a safe, read-only ClickHouse query, runs it, and picks a chart. This is the judges' own "ask my logs anything" idea, pointed at a real civic dataset. The generated SQL and the run time stay on screen, so every answer is auditable.

Every result shows its query latency, so the speed is proven, not claimed.

Trigger.dev runs the whole show

The experience is a Trigger.dev chat.agent(). The agent routes each message to tools that query ClickHouse and return visual specs (a map, a chart, a card), never prose.

But it does more than chat:

Durable, long-running ingestion tasks stream millions of rows into ClickHouse, retryable, no timeouts.
A durable child sub-task runs the "should you appeal?" debate: two advocates argue in parallel, then a verdict. Even if the browser closes, it keeps running.
A scheduled cron task re-checks saved homes and flags changes, surfaced by a "Watch this home" button in the UI.

The bonus round: OLTP + OLAP in one query

For the OLTP + OLAP category, saved appeals live in Postgres. The portfolio page runs a single ClickHouse query that pulls those Postgres rows in via the postgresql() table function and joins them against the analytics tables. One statement, two engines, with the query and its latency shown right on the page.

Why it matters

This is not a toy. In Cook County alone we measure about $460 million a year of unfair over-assessment, live over real homes. Scaled nationally, and consistent with published research across 118 million homes, that points to roughly $20 billion a year, quietly taken from the people who can least afford it.

Overtaxed turns those buried public records into a verdict, visible evidence, and a path to act, in about ten seconds.

Try it

Live: https://overtaxed-mauve.vercel.app
Code (MIT): https://github.com/usv240/overtaxed

Built by team Agent Forge for the ClickHouse x Trigger.dev Virtual Summer Hackathon 2026. Thanks for reading.

I built a spell-checker for the statistics in research papers

Ujwal Vanjare — Fri, 03 Jul 2026 16:49:52 +0000

I kept running into the same uncomfortable fact. Somewhere around half of published
psychology papers have at least one number that doesn't add up. It's usually not
fraud. It's tired people making small mistakes. A p-value typed wrong. An average
that can't exist for the sample size. A result called "significant" that, when you
actually check it, isn't.

The strange thing is that these slip through. Reviewers read for the ideas and the
argument, not the arithmetic. Almost nobody sits down and recomputes a p-value by
hand. And the tools that can do it are expensive, closed, and sold to journals, so
they only run after you've already submitted. The one person who'd benefit the most,
the author about to hit submit, gets nothing.

So I built Rigor. You paste your paper, or drop in a PDF, and it checks the numbers
for you in a few seconds.

What it checks

Six checks now, and they each catch a different kind of slip.

It recomputes every p-value from the test statistic. If you wrote p < .001 but the
math says p = .06, it tells you, and it flags the ones that flip a result from
significant to not.

It runs the GRIM and GRIMMER tests. GRIM catches an average that can't exist: the mean
of ten whole-number answers can't be 3.45. GRIMMER does the same for the standard
deviation. Small checks like that catch typos, and sometimes worse.

It compares the degrees of freedom to the sample size. If a test needs more people
than the study actually collected, something is mislabeled.

It recomputes a reported effect size (Cohen's d) from the t-statistic, and flags a
mismatch that rounding can't explain.

And it reads the claims, checking whether the words match the numbers, like calling a
result significant when the recomputed value says it isn't.

Every finding comes with a plain explanation and a suggestion for what to do next,
not just a red mark.

The part I'm proudest of

Most tools stop at "something is wrong." Rigor goes one step further and tells you
which single number is most likely the mistake. A paper's statistics are all linked,
the sample size, the degrees of freedom, the test statistic, the p-value, the mean. So
when several checks fail, Rigor looks for the one correction that would resolve the
most of them, and proves it by re-running the checks with that value substituted. On
the demo paper it says something like "the sample size is the likely typo; fix it and
every flagged test lines up." That is the difference between finding a problem and
knowing how to fix it.

It also puts each paper in context. Every audit compares it to a published baseline
(about one in ten reported p-values is inconsistent, from a study of roughly 250,000 of
them), and estimates the time it just saved you against checking by hand.

The question I was afraid of

Early on I kept asking myself whether this was just ChatGPT with a nice screen on
top. That question kills a lot of AI projects, and it deserved a real answer.

Here's how I made sure it wasn't. Qwen, the model, only reads. Its one job is to pull
the numbers and claims out of messy writing. It never decides if something is right
or wrong. Every verdict comes from plain math, exact statistics that can't be made up.
You can even prove it: turn the AI off and run the math engine over 530 test cases, and
it still scores 100 percent. If the model misreads something, the worst case is a flag
you can dismiss. It can never invent a wrong answer, because it never gives an answer at
all. The math does.

That split, where the model reads and the math judges, is the whole idea.

Where Qwen came in

I used Qwen through Alibaba Cloud's Model Studio. The part only a good model can do is
read real, free-form scientific writing and pull the statistics out as clean data. I
used Qwen's function calling for this, so instead of hoping it hands back tidy text I
can parse, it fills in a proper structured form.

Because reading is the only uncertain step, I also let it read a paper a few times and
keep only the numbers the runs agree on, and it shows that agreement as a score. It
turns the one shaky part into a measured number.

Turning it into a real agent

For a while Rigor was a straight line. Extract, check, report. Useful, but that's a
pipeline, not an agent. So I gave Qwen the checks as tools and let it run its own loop.
Now it reads the paper, decides what to check, calls the tools itself, thinks about the
results, and then tells you whether the problems look like one-off typos or something
more systematic.

My favorite part is that you can watch it happen. There's a live log in the app that
streams each step as it goes: reading, calling grim_test, calling recompute_pvalue,
reasoning, then the verdict.

The mistake that taught me the most

I ran an early version on some real papers, feeling pretty smart, and it flagged a
measurement of 6.07 micrometres as an impossible average. That's nonsense. GRIM only
applies to whole-number ratings, not physical measurements. My extraction was too
eager. I tightened it so it only checks what it should, and now it stays quiet when a
paper is out of its depth. Being careful turned out to matter as much as being clever.

Does it work on real papers?

Yes, and I have a favorite example. I ran it on a published geology paper about rock
erosion. It flagged a correlation the authors reported "across the four rock types" as
significant. With only four data points that is a shaky claim, and it recomputes to
about p = 0.08. I read the paper's own methods to be sure, and yes, four rock types,
four points. Rigor caught it on its own.

Then I ran it over 26 real published papers in a single pass, the editorial-review
shape. Recomputing all their reported statistics by hand would take about three hours.
Rigor did it in under six minutes and narrowed the pile to the three papers that needed
a human look, ranking the one I had verified worst of all.

It's honest about its limits, too. On long, messy papers extraction can be noisier,
which is exactly why there's a human review step and the agreement score. It flags, you
decide, and you dismiss anything it got wrong before you export the report.

Running it on Alibaba Cloud

The whole thing runs on Alibaba Cloud, in a container on a small server in Singapore,
with Qwen doing the reading. It's open source, and it goes where you work: a website, a
command-line tool, a GitHub Action that screens papers on every push, and an MCP server
so other AI agents can call the checks too. Rigor ended up being a small building
block, not just a website.

What I'd pass on

If you build with a model, use it for the one thing only it can do, which is making
sense of human writing, and let ordinary code own anything that has to be correct. That
line is what turns a demo into something people can actually trust. And for a tool about
integrity, being honest about what it can't do, and only ever showing numbers I can
prove, made it more credible, not less.

Rigor is free and open source. The code is at https://github.com/usv240/rigor, and you
can try the live version, paste a paper, and watch it work.

Making product recalls executable with Aurora DSQL and Vercel

Ujwal Vanjare — Thu, 25 Jun 2026 03:53:41 +0000

Live demo: https://safestate.vercel.app , code: https://github.com/usv240/safestate

A product recall today is basically a notice. It lives on a webpage, or a PDF, or an email that somebody is supposed to read. Say the problem out loud and it gets uncomfortable fast. A recalled crib can be listed and sold to another family, and nobody in that sale ever sees the recall. Reselling recalled goods is actually illegal, and recalled infant products have killed kids.

I spent this hackathon building something to close that gap. I called it SafeState, and the idea is small: make the recall do something. When a second-hand item is listed or sold, the marketplace checks SafeState first, and recalled units get blocked right at checkout. It is precise down to the serial number, so safe units still sell.

It runs on the stack this hackathon is about. A Next.js front end on Vercel, with Amazon Aurora DSQL behind it.

Why DSQL is the whole point here

The promise SafeState has to keep is this: the moment a recall lands in any region, no marketplace anywhere should ever read that product as "safe" again.

That is a strong consistency problem, not a nice-to-have. If there is any window where a recalled product still looks safe, that is exactly when it gets sold. An eventually consistent store or a nightly sync leaves that window open. DSQL's active-active, multi-region setup with strong consistency is what closes it.

I set up a real peered cluster across us-east-1 and us-east-2, with us-west-2 as the witness. Write a recall through one region's endpoint and you can read it back from the other region right away. There is a page in the app that lets you run that yourself.

The one trick that makes it work

DSQL runs on snapshot isolation (PostgreSQL REPEATABLE READ) with optimistic concurrency. It catches write-write conflicts at commit time. Snapshot isolation will not protect you from write skew, so I had to design around that.

To guarantee that a recall and a sale of the same product actually collide, I make both of them write the same row. Every model has one safety_guard row that holds its status and an epoch number.

// authorize-transfer, simplified. The AUTHORIZED path touches the SAME guard
// row a concurrent recall writes, so DSQL is forced to detect the conflict.
await client.query("BEGIN");
await client.query("SELECT epoch FROM safety_guard WHERE model_id = $1", [modelId]);

// ...evaluate every active directive against THIS unit's serial...
// if it is covered, return BLOCKED. otherwise:

await client.query("INSERT INTO ownership_transfers (...) VALUES (...)");
await client.query("UPDATE product_instances SET current_owner_id = $1 WHERE id = $2", [buyer, id]);
await client.query("UPDATE safety_guard SET updated_at = now() WHERE model_id = $1", [modelId]); // the conflict-forcing write
await client.query("COMMIT"); // the loser throws SQLSTATE 40001 / OC000 here

If the recall commits first, the sale's COMMIT throws SQLSTATE 40001 (OC000). A small wrapper catches it, backs off with some jitter, and runs the whole transaction again. The second time around it reads the recalled state and returns BLOCKED. So there is no version of events where a recalled product slips through as safe.

const RETRYABLE = new Set(["40001", "OC000", "OC001"]);
// retry the WHOLE transaction on conflict, backoff plus jitter, max 3 attempts

Proving it under load

A guarantee you cannot see is just a claim, so I put a stress test right in the app: a hundred concurrent attempts to buy a recalled unit, fired at the live cluster at once. Every one comes back blocked. Zero recalled units sell, no matter the concurrency.

Getting there taught me something. My first version put a SELECT ... FOR UPDATE on the guard row in every check. That was overkill. Two blocked checks on the same model would each take a write intent on that one row and conflict with each other for no reason. The conflict I actually care about is between a recall and an authorized sale, and both of those already write the guard row. So I dropped the FOR UPDATE from the read. The blocked path stopped fighting itself, the load test went clean, and the recall versus sale conflict still fires exactly as before.

Two databases, on purpose

Not everything belongs in the transactional store. Every public check, verify, and scan is an event worth counting, and every Safety Receipt is a small durable record. That stream is write-heavy and key-accessed, and it does not need a distributed transaction. So it lives in Amazon DynamoDB, while Aurora DSQL keeps the transactional core. Picking the right database per workload, instead of forcing one to do both, kept the hot path clean.

The Vercel side

Route handlers talk to DSQL over the normal Postgres protocol, but auth is a short-lived IAM token minted per connection with @aws-sdk/dsql-signer. There is no database password sitting in an env var anywhere.

A Vercel Cron job pulls real recalls from the public CPSC API once a day. And Claude reads messy second-hand listings, the kind a person actually writes ("used baby sleeper, works fine"), and figures out which recall they match, with a confidence score. The uncertain ones go to a review queue instead of being auto-blocked.

Two more things the same app does. When a recall is issued, it walks live ownership and emails the people who own one now, not the original buyers. And the public check fans out to CPSC, FDA, and NHTSA at once, so you can look up a product, a bag of spinach, or a car.

One thing that cost me an hour. Vercel functions run on Lambda, and Lambda reserves AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION. You cannot set those as env vars. So I pass the DSQL credentials under different names and hand them to the signer directly.

const creds = process.env.SAFESTATE_AWS_ACCESS_KEY_ID
  ? { accessKeyId: process.env.SAFESTATE_AWS_ACCESS_KEY_ID, secretAccessKey: process.env.SAFESTATE_AWS_SECRET_ACCESS_KEY }
  : undefined; // local dev falls back to the default AWS provider chain
const signer = new DsqlSigner(creds ? { hostname, region, credentials: creds } : { hostname, region });

A few things that helped

If you build on DSQL, pick a problem where being correct under concurrency is the actual product, not a side detail. That is where it earns its keep. Make your conflicting operations write the same row so OCC has something to catch. And write the retry-on-40001 wrapper before anything else, because you will lean on it constantly.

Recalls should stop being PDFs and start being decisions. Aurora DSQL and Vercel got me there over a weekend.

Live: https://safestate.vercel.app , code: https://github.com/usv240/safestate

I built this for the H0: Hack the Zero Stack hackathon. #H0Hackathon

I built a GitLab flow that tells you what a diff actually means

Ujwal Vanjare — Mon, 22 Jun 2026 23:08:24 +0000

Every code review starts the same way. You open a diff and try to figure out
what it actually means - not just what changed, but what depends on it,
whether it breaks something downstream, and whether someone else is about to
step on the same code. That investigation usually falls on the reviewer,
by hand, every single time.

I spent the last two weeks building something to change that.

What I built

Orbit Change Passport is a GitLab Duo Agent Platform flow that fires
automatically when a merge request is marked ready for review. It queries
GitLab Orbit's Knowledge Graph and posts a structured comment before anyone
reads a single line of the diff.

Here is what the comment covers:

Changed Surface - the exact functions and methods the diff touched, identified by name, not line numbers
Dependency graph - a live Mermaid diagram of everything that imports the changed modules, rendered inline in GitLab
Conflict Radar - every other open MR right now that touches the same code, caught before merge instead of at it
Cross-project blast radius - files in other projects in the group that depend on what changed. This one is only possible because Orbit's graph spans the whole group, not just one repo
Test gaps - test files that import the changed module but were not touched in this MR
Suggested reviewers - based on recent authorship of dependent files
Reviewer Brief - a second shorter comment posted immediately after: one paragraph with the single most important thing before reading the diff

A companion Duo Chat agent lets reviewers keep asking questions against the
same graph after the passport posts - "what else calls this?",
"who imports this file?" - answered live, not restated from the comment.

A concrete example

Here is what the impact line looks like on a real run:

Impact: HIGH - 2 definitions changed - 2 within-project dependents -
cross-project blast radius (3 projects) - 1 conflict

That single line tells me: this change has dependents outside this repo,
and another open MR is already touching the same functions. Both of those
things would normally take 10 minutes to find manually. They showed up
automatically before I opened the diff.

The cross-project result is the one that surprised me most. A docstring
change to engine/orbit_client.py surfaced 14 dependent files across three
other projects in the group. Nobody catches that by scrolling through a diff.

How it works

The flow uses a three-tier strategy to identify which functions a diff
actually touched:

DEFINES traversal in the Orbit graph (File to Definition edge)
fqn-fragment query as a fallback for Rust crates
Bounded page scan as a last resort

Each tier produces a different confidence level in the output. This redundancy
matters because query availability on the live Orbit instance is not guaranteed.
DEFINES, IMPORTS, CALLS, and Definition filtering have each independently been
unavailable within a two-hour window during testing. A flow that only works
when one specific query is live is not useful.

For dependents, the flow runs ImportedSymbol lookups in both FQN and
crate-relative forms for Rust, and by identifier_name stem for Python files.
That last one took a while to figure out. Python relative imports
(from . import module) are stored in Orbit with import_path set to the
package name and identifier_name set to the module stem, not as a dotted
path. We found that by reading the actual graph data.

The Reviewer Brief is a second call to create_merge_request_note within the
same agent turn. We arrived at this after two other approaches failed: a
router-chained second AgentComponent that never started, and a second ambient
flow that turned out to share the trigger silently with the first. Only the
most-recently-enabled one fires - that was not documented anywhere and took a
while to diagnose.

What is in the repo

flows/change-passport/change-passport.yaml   - the Duo Agent Platform flow
skills/ask-orbit-passport/SKILL.md           - companion Duo Chat agent
engine/passport_runner.py                    - CLI runner, posts to any MR
engine/orbit_client.py                       - Orbit query layer
research/findings.md                         - every query pattern confirmed
                                               against the live graph

Try it

The flow is live in the AI Catalog. The repo is public and MIT licensed.

Built for the GitLab Transcend Hackathon.

A diff shows what changed. Orbit Change Passport shows what that change means.

Scaling Astrolord: Our Journey with Serverless on AWS

Ujwal Vanjare — Wed, 04 Feb 2026 03:23:53 +0000

When we started building Astrolord, we knew we had a tricky engineering problem on our hands. We were building an AI-powered astrology platform that needed to perform heavy astronomical calculations one second and stream personalized AI responses the next. We needed infrastructure that could handle a sudden spike in traffic, like during a major planetary transit, without burning a hole in our pocket during quiet hours.

We decided early on to go all-in on AWS Serverless. It wasn't just about avoiding server management; it was about building an architecture that could scale to zero when no one was using it, and scale up infinitely when they were. Here is a look at how we pieced it all together.

The Compute Layer: AWS Serverless

The heart of our backend is Python. The challenge was deploying our application in a way that didn't require maintaining a fleet of servers or paying for idle time.

We chose AWS Lambda for its efficiency. By adapting our web application to run in a serverless environment, we can write modern, clean code while letting AWS handle the underlying infrastructure.

There were trade-offs, of course. We had to be careful with "cold starts", the initial delay when a function wakes up. We spent time optimizing our application startup to keep latency minimal. But the benefit of paying literally zero dollars when no traffic is hitting the API made it an easy choice.

The Front Door: API Gateway

Sitting in front of those Lambda functions is Amazon API Gateway. Connective tissue is often overlooked, but for us, this piece is critical. It acts as our secure entry point, routing requests to the correct backend services.

Beyond just routing, we rely on it for stability. We configured usage plans and throttling rules directly at the gateway level. This means if a bad actor tries to hammer our API, AWS blocks them before they even wake up our compute layer, saving us money and processing power.

Delivering the UI: S3 and CloudFront

For the frontend, we built a static React application. We didn't want to run a web server or manage containers just to serve HTML and JavaScript files.

Instead, we use what I consider the "gold standard" for static hosting: Amazon S3 paired with CloudFront. We upload our build artifacts into an S3 bucket, which offers incredible durability. Then, CloudFront sits in front of that bucket, caching our application at edge locations all over the world.

The result is that a user in Mumbai loads the app just as fast as a user in New York. We also configured strict security controls, ensuring that no one can bypass the CDN to hit our bucket directly.

Observability

One thing they don't tell you about distributed systems is that debugging can be a nightmare if you aren't prepared. Since we don't have a single server to SSH into, we rely heavily on centralized logging and monitoring.

We capture all standard output from our functions into CloudWatch Logs. We also set up specific metric filters to track errors and throttling events. If our error rate crosses a certain threshold, an alarm triggers and we get notified immediately. It gives us the confidence to deploy on a Friday (well, almost).

The Economics of Serverless

One of the biggest wins for us wasn't just technical, it was financial.

With a traditional EC2 or container-based architecture, you are paying for capacity 24/7. Even if no one visits your site at 3 AM, that server is still running, and you are still receiving a bill.

With this serverless setup, our infrastructure cost aligns perfectly with our business growth.

Zero Idle Costs: When our app is quiet, our compute bill is literally $0.
Generous Free Tier: AWS offers a substantial free tier for Lambda and API Gateway, meaning we effectively run our development and testing environments for free.
No Over-Provisioning: We never have to guess how many servers we need. The system just scales to meet demand, whether that's 5 users or 5,000.

Why This Architecture Works for Us

Usage patterns in our app are unpredictable. Some days are quiet; other days, everyone wants to know what the full moon means for them.

This serverless architecture on AWS absorbs that volatility perfectly. We don't have to provision for peak capacity and pay for idle time. We just pay for the milliseconds of compute we actually use. It has allowed us to focus less on "keeping the lights on" and more on improving the actual product.

Time-Travel Debugging for Python: A Complete Tutorial

Ujwal Vanjare — Fri, 23 Jan 2026 05:10:45 +0000

Time-Travel Debugging for Python: A Complete Tutorial

Building web applications means dealing with external APIs, databases, and the inevitable production bugs. I'm going to show you how to capture production issues and debug them locally without ever hitting those external services again.

This is a complete walkthrough using Timetracer with a Starlette application. By the end, you'll have a working example and understand how to apply this to your own projects.

Note: If you're new to Timetracer, you might want to check out my initial post about why I built this tool, or the v1.4 release post covering Django and pytest integration.

This tutorial focuses specifically on Starlette integration and shows the complete debugging workflow with the new v1.6.0 dashboard features.

The Problem

You know that moment when a bug happens in production? You spend hours trying to reproduce it locally. You're making API calls to third-party services, dealing with rate limits, stale data, and that nagging feeling you're not testing the exact scenario that failed.

Traditional debugging flow:

Bug reported in production
Try to reproduce locally (often fails)
Add logging and redeploy (slow)
Hope you captured enough context (usually didn't)
Repeat until fixed (hours or days)

There's a better way. With Timetracer, you capture the entire request context in production and replay it locally. Think of it as a flight recorder for your web application.

Setting Up the Project

Let's build a simple API that proxies GitHub user data. First, install the dependencies:

pip install starlette uvicorn httpx timetracer

Create a file called app.py:

from starlette.applications import Starlette
from starlette.routing import Route
from starlette.responses import JSONResponse
import httpx

async def homepage(request):
    return JSONResponse({
        "message": "Welcome to the Starlette + Timetracer example",
        "endpoints": ["/", "/user/{username}", "/repos/{username}"]
    })

async def get_user(request):
    username = request.path_params["username"]
    async with httpx.AsyncClient() as client:
        response = await client.get(f"https://api.github.com/users/{username}")
        return JSONResponse(response.json())

async def get_repos(request):
    username = request.path_params["username"]
    async with httpx.AsyncClient() as client:
        user_resp = await client.get(f"https://api.github.com/users/{username}")
        user_data = user_resp.json()

        repos_resp = await client.get(f"https://api.github.com/users/{username}/repos")
        repos = repos_resp.json()

    return JSONResponse({
        "username": username,
        "total_repos": user_data["public_repos"],
        "top_repos": [{"name": r["name"], "stars": r["stargazers_count"]} 
                      for r in sorted(repos, key=lambda x: x["stargazers_count"], reverse=True)[:5]]
    })

app = Starlette(debug=True, routes=[
    Route("/", homepage),
    Route("/user/{username}", get_user),
    Route("/repos/{username}", get_repos),
])

This gives us three endpoints: a homepage, a user lookup, and a repo list. The last two hit the GitHub API.

The Starlette application with three endpoints

Integrating Timetracer

Now add Timetracer. Import the integration and call auto_setup():

from timetracer.integrations.starlette import auto_setup

# ... your routes ...

app = Starlette(debug=True, routes=[
    Route("/", homepage),
    Route("/user/{username}", get_user),
    Route("/repos/{username}", get_repos),
])

# This is the only line you need for Timetracer
auto_setup(app, plugins=["httpx"])

That's it. One line of code. This adds middleware that captures every request and tracks all httpx calls to external APIs.

Recording Requests

Start the server in record mode:

export TIMETRACER_MODE=record
uvicorn app:app --reload

Your terminal should show Timetracer capturing requests:

Terminal output showing Timetracer recording requests with timing information

Now let's make some requests and see what gets captured.

Request 1: Homepage

curl http://localhost:8000/

Response:

{
  "message": "Welcome to the Starlette + Timetracer example",
  "endpoints": ["/", "/user/{username}", "/repos/{username}"]
}

Browser showing the homepage JSON response

Terminal output:

timetracer [OK] recorded GET /  id=cddb  status=200  total=9ms  deps=none
  cassette: cassettes/2026-01-23/GET__root__cddb6be9.json

Notice deps=none because this endpoint doesn't make any external calls.

Request 2: User Lookup

curl http://localhost:8000/user/octocat

Response:

{
  "login": "octocat",
  "name": "The Octocat",
  "bio": null,
  "public_repos": 8,
  "followers": 21594
}

GitHub user data returned through our API

Terminal output:

timetracer [OK] recorded GET /user/octocat  id=88d7  status=200  total=472ms  deps=http.client:1
  cassette: cassettes/2026-01-23/GET__user_octocat__88d76871.json

This time deps=http.client:1 shows one external HTTP call was tracked. The duration is 472ms instead of 9ms because we're waiting for GitHub's API.

Request 3: Repository List

curl http://localhost:8000/repos/octocat

Top repositories for the octocat user

This endpoint makes two GitHub API calls: one for the user data and one for the repository list.

What Got Saved?

Each cassette is a JSON file containing your request, response, and all external dependencies with timing information.

Cassette file showing the captured request, response, and external API calls

The cassette includes:

Request details (method, path, headers, body)
Response details (status, headers, body, duration)
All external dependencies (each GitHub API call with its own timing)
Metadata about the session (framework, timestamp, etc.)

Using the Dashboard

Now for the interactive part. Timetracer includes a web dashboard to browse and analyze your captured requests.

Start the dashboard server:

timetracer serve --dir cassettes --port 3000

Open http://localhost:3000 in your browser:

Dashboard showing all captured requests with statistics

The dashboard shows:

Total requests, success count, error count
Every captured request with method, path, status, duration, and dependencies
Search and filter capabilities
View details or replay any request

Viewing Request Details

Click "View" on the /repos/octocat request:

Detailed view showing request, response, and external API dependencies

The detail view shows:

Request metadata: Path, method, timestamp
Response: Status 200, duration 524ms
Dependency Events: Both GitHub API calls with individual timings
- GET https://api.github.com/users/torvalds (104ms)
- GET https://api.github.com/users/torvalds/repos (95ms)
Ready-to-use replay command

This view tells you exactly what happened during the request, including all external services that were called.

Filtering Requests

Type "repos" in the search box:

Dashboard filtered to show only repository-related requests

The dashboard now shows "Showing 5 of 19 cassettes" with only the matching requests visible.

You can also filter by HTTP method or status code to focus on specific types of requests.

Inspecting the Raw Data

For technical inspection, the dashboard includes a Raw JSON viewer:

Raw JSON view showing the complete cassette structure

This gives you direct access to the underlying cassette data, making it easy to verify exactly what state is being captured and will be replayed.

Debugging a Real Bug

Now let's use Timetracer for what it's really good at: debugging production issues without touching production.

The Bug Appears

Imagine a user reports that requesting a non-existent GitHub user crashes the server with a 500 error. The problematic code:

async def get_user(request):
    username = request.path_params["username"]
    async with httpx.AsyncClient() as client:
        response = await client.get(f"https://api.github.com/users/{username}")
        return JSONResponse(response.json())  # Crashes on 404

When someone requests a user that doesn't exist, GitHub returns 404, but our code assumes success and tries to parse the error response.

Even though the app crashes, Timetracer still captures the request:

timetracer [ERROR] recorded GET /user/nonexistent-user-12345  id=bad1  status=500  total=156ms  deps=http.client:1
  cassette: cassettes/2026-01-23/GET__user_nonexistent-user-12345__bad1234.json

Inspecting the Error

In the dashboard, click "View" on the failed request:

Dashboard detail view showing a 404 error from GitHub API

The detail view clearly shows that GitHub returned a 404, which propagated to our endpoint as a 500 error. You can see exactly what happened: the external API call failed, and our code didn't handle it properly.

The Fix

Looking at the dashboard detail view, you can see GitHub returned 404. Fix the code:

async def get_user(request):
    username = request.path_params["username"]

    async with httpx.AsyncClient() as client:
        response = await client.get(f"https://api.github.com/users/{username}")

        # Check status code before parsing
        if response.status_code == 404:
            return JSONResponse(
                {"error": "User not found"},
                status_code=404
            )

        return JSONResponse(response.json())

Getting the Replay Command

The dashboard provides a ready-to-copy replay command:

Ready-to-use replay command for testing the fix

Just copy this command to test your fix with the exact scenario that failed in production.

Testing the Fix Without Network

This is where Timetracer shows its real value. You can test the fix using the captured cassette without making any real API calls to GitHub.

Stop the server and restart in replay mode:

export TIMETRACER_MODE=replay
export TIMETRACER_CASSETTE=cassettes/2026-01-23/GET__user_nonexistent-user-12345__bad1234.json
uvicorn app:app --reload

Server running in replay mode with mocked external API responses

Make the same request:

curl http://localhost:8000/user/nonexistent-user-12345

Terminal shows:

timetracer replay GET /user/nonexistent-user-12345  mocked=1  matched=OK  runtime=5ms

Response:

{
  "error": "User not found"
}

Status: 404

The fix works. Notice:

No network call - the response came from the cassette
Fast: 5ms instead of the original 156ms
Exact scenario: Same 404 from GitHub that caused the original crash
Offline: This works with no internet connection

You just debugged and fixed a production bug without touching production or making a single external API call.

Performance Comparison

Let's compare the timing differences:

Record Mode vs Replay Mode

Endpoint	Record Duration	Replay Duration	Speedup
`/`	9ms	8ms	1.1x
`/user/octocat`	472ms	8ms	59x faster
`/repos/octocat`	524ms	10ms	52x faster

Comparison of request durations in record mode versus replay mode

For endpoints without external calls, the times are similar. But anything that touches an external API or database becomes dramatically faster in replay mode.

This isn't just about speed. It's about reliability. Tests that depend on external APIs can be flaky due to network issues, rate limiting, or changing data. Replay mode eliminates all those problems.

When to Use This

I've found Timetracer most useful in these scenarios:

1. Debugging Production Bugs

When a user reports an issue, capture the failing request in production. Download the cassette and debug locally with the exact same conditions. No need to reproduce complex scenarios or guess at what data caused the problem.

2. Integration Testing

Tests that hit real APIs are slow and unreliable. Record your test scenarios once, then replay them. Tests run in milliseconds instead of seconds, and they never fail due to network issues or rate limiting.

3. Offline Development

Working on a plane or anywhere without internet? Load up cassettes with the API responses you need. Everything works normally without network access.

4. Performance Analysis

The dashboard shows you exactly how long each external dependency takes. If your endpoint is slow, you can see whether it's your code or a slow external API.

5. Preventing Regressions

When you fix a bug, keep the cassette and add it to your test suite. That specific scenario is now covered forever.

Framework Support

Timetracer works with:

Supported web frameworks and external service integrations

Web Frameworks:

FastAPI
Starlette (new in v1.6.0)
Flask
Django

External Services:

httpx and requests (HTTP clients)
Motor and PyMongo (MongoDB)
SQLAlchemy (SQL databases)
Redis

The integration is similar across all frameworks. Usually just auto_setup(app) or adding middleware.

Trade-offs to Consider

Storage: Each cassette is a JSON file. If you have many unique requests, you'll accumulate files. Clean up old cassettes periodically or store them in S3.

Sensitive data: Cassettes contain your actual request and response data. Review what's being captured, especially in production. Timetracer has built-in redaction for common sensitive fields like passwords and tokens, but verify this for your use case.

Cassette maintenance: API responses change over time. You'll need to re-record cassettes when your external dependencies change their response format.

Not a replacement: This isn't trying to replace your testing framework or mocking library. It's a debugging tool that captures production context and lets you work with it locally.

Getting Started

Install Timetracer:

# For Starlette
pip install timetracer[starlette]

# For FastAPI
pip install timetracer[fastapi]

Integrate into your app:

from timetracer.integrations.starlette import auto_setup
auto_setup(app, plugins=["httpx"])

Run in record mode:

export TIMETRACER_MODE=record
uvicorn app:app

View the dashboard:

timetracer serve --dir cassettes --port 3000

Test in replay mode:

export TIMETRACER_MODE=replay
export TIMETRACER_CASSETTE=path/to/cassette.json
uvicorn app:app

Conclusion

The workflow I showed here - capturing a failing production request, viewing it in the dashboard, fixing the bug, and testing the fix in replay mode - saves hours compared to traditional debugging.

Instead of:

Trying to reproduce the bug
Adding logging
Redeploying
Hoping you captured enough context
Repeating until fixed

You can:

Download the cassette
View it in the dashboard
Fix the code
Verify the fix in replay mode
Deploy with confidence

The complete example code is on GitHub at github.com/usv240/timetracer. All 174 tests are passing, and version 1.6.0 just added Starlette support and PyMongo integration.

If you work with external APIs, spend time debugging production issues, or want faster integration tests, give it a try.

Resources

GitHub: https://github.com/usv240/timetracer
PyPI: https://pypi.org/project/timetracer
Documentation: GitHub README
Example code: See examples/starlette_example/ in the repo

Tags: #python #starlette #fastapi #debugging #testing #devtools