DEV Community: Victor Isaac

Ten builds since July, every one dated

Victor Isaac — Wed, 21 Oct 2026 20:19:21 +0000

This is the index for my series Agents with receipts: everything I have built since July 2026, in order, with the date, what it was for, how it did, and where it stands today. I will keep it updated as builds finish.

A thread runs through most of these: the model is allowed to plan and explain, but the decisions that matter are made in code. Budgets, safety gates, scores and verdicts live in plain functions you can test. The receipts are the tests, the benchmarks and the limits sections.

The timeline

Date	Build	What it is	Event	Result	Status
12 Jul 2026	Exam Guard	Exam paper fingerprints on the Cardano testnet	IndiaCodeX'26, 12 hours, Team Pragmatic	Did not place	Done. Post
22 to 23 Aug 2026	Kitchen Agent	Procurement agent for institutional kitchens, with a budget it cannot override	LaunchPadX 2026, SNIST Hyderabad, Team Rudra	1st of 231 teams	Done. Repo
12 Sep 2026	Basket Buddy	Self-learning browser agent that keeps a notebook per site	SLAB Hackathon, CMR Institute of Technology, solo, one day	2nd place	Done. Post
Sep 2026	Antibody	Root cause analysis for microservices, allowed to say "insufficient evidence"	Microsoft Codathon, Problem 3, team of five	1st in college	Done. Post
25 to 26 Sep 2026	JAMES	Touch-free maintenance copilot with a safety gate that has no AI in it	AgentX-2026, TKREC Hyderabad	1st prize	Shipped, still improving. Post
28 Sep 2026	Prequal	Vendor questionnaire agent that answers from memory and never invents a certificate	Team build, five people, memory on Hindsight	Shipped	Done. Post
29 Sep 2026	Procurement Pilot: Bid Compliance	Fifteen checks against ten registries; code decides, AI explains, an officer signs	Smart India Hackathon 2026, SIH26100, Team Ace Azael	Submitted	Live demo. Post
18 Oct 2026	Ithamar	Checks every row of a bank statement against its own running balance, in the browser	Side build	Live	Still improving. Post
21 Oct 2026	Sapor	Group taste agent: one real place for the whole table, ranked by the person it suits least	Qloo Agentic Hackathon	Due 31 Oct	Building now. Post
21 Oct 2026	ATLAS Live	Hologram studio in the browser: pull machines apart with your hands	Nebius x NVIDIA Global AI Hackathon	Due 30 Oct	Building now. Post

In numbers

10 builds since 12 July
4 podium finishes: 1st at LaunchPadX 2026, 1st at AgentX-2026, 2nd at SLAB, 1st in college at the Microsoft Codathon
2 in progress right now, both due before the end of October

Where to start

If you read one post, read JAMES: it is the clearest example of a gate written in code that the model cannot talk its way past. If you like numbers, Antibody and Ithamar are the two with the most careful numbers behind them.

Everything is open source on GitHub, and the portfolio lives at victorisaac.vercel.app.

I wrote this with AI assistance. Dates come from each repo's history and each event's own records.

Rank every place by the person it suits least: building Sapor, a group taste agent

Victor Isaac — Wed, 21 Oct 2026 20:17:58 +0000

Status: building now. Started 21 Oct 2026 for the Qloo Agentic Hackathon. Submission due 31 Oct 2026. Live: sapor-nu.vercel.app

"Where should we go?" is the most asked and worst answered question in every group chat. Ratings tell you what is popular, not what suits these particular people. Usually the loudest person picks, and someone sits through a meal that was never for them.

People are bad at describing their food taste in the abstract and very good at naming things they love. So Sapor (Latin for taste) starts there:

Everyone names a film, show or artist they love. Sapor reads the taste behind those picks with Qloo, finds one real place nearby that suits the whole table, and says what each person should order there, with the evidence for every line.

Rank by the person it suits least

The interesting decision in Sapor is not the AI. It is one line of arithmetic.

My first version ranked venues by Qloo's affinity score for the whole table. It picked a venue Qloo loved even when one guest matched it at 35%. Averages hide the person who is left out. So the ranking now looks at the weakest seat first:

table score = 0.60 × weakest person's match
            + 0.25 × average match
            + 0.15 × Qloo affinity for the venue

A match (0 to 1) is how much of one person's Qloo taste profile, cuisines and kinds of room, the venue's own Qloo tags carry. It is not a probability that they will enjoy it, and the app says so.
Qloo's affinity sits in a narrow band among nearby candidates, so at 15% it can only settle near-ties. It can never outweigh a 25-point gap in the weakest match.
Ties go to the better weakest match, then the better average, then the name, so the same input always gives the same answer.

The 35% counterexample is a unit test now, next to a strict maximin baseline.

A real run

The sample table: Riya loves BTS and Squid Game. Arjun loves A. R. Rahman and Pushpa, and is vegetarian. Meera loves Prateek Kuhad and Before Sunrise. Coffee, in Jubilee Hills, Hyderabad.

Qloo reads very different crowds behind those picks. Riya's leans Korean, Thai and Chinese, in hip rooms. Arjun's leans Indian, in comfortable rooms. Meera's leans Indian, in serene ones.

The search returned 30 venues; 2 were set aside because they had no food data at all (Qloo's cafe filter sometimes returns gyms). Terra Cafe & Bistro won with Riya at 99, Arjun at 100 and Meera at 83: weakest 83, average 94. The runner-up had a higher Qloo affinity, but its weakest match was 74, so it lost.

Then the orders, from Terra's own menu highlights in Qloo: Korean chicken bao for Riya, pav bhaji bruschetta for Arjun (vegetarian by name, flagged "confirm with staff"), paneer Korean bao for Meera.

Guardrails, again in code

Ambiguity stops the agent. "Dune" is a film, a book and a German techno group, each drawing a different crowd. If two candidates are close, Sapor asks instead of guessing.
Vegetarians are protected by name. A vegetarian seat only gets dishes whose names carry no meat, fish or egg.
The one-line explanations are validated. Groq writes a short reason per person from the evidence alone. A validator rejects any line that addresses the reader, praises food with words not in the data, or leaks a seat letter, and falls back to a template.
Privacy by design. Names never leave the browser. The server sees seats as A, B and C, and Qloo only ever receives entity ids, tag ids and a place name.

A full plan costs about one Qloo call per person plus one venue search, with a six-hour cache for repeats.

Still to do before 31 Oct

The demo video and the submission write-up
More test tables, because the link from a dish name to a cuisine is my own keyword map, not Qloo data, and that is the part most likely to be wrong

Repo (MIT): github.com/victorisaacchinta-eng/sapor

I wrote this with AI assistance, working from the repo. The run above is the app's sample table, measured with live Qloo calls on 21 Oct 2026.

My first hackathon build: an exam paper's fingerprint on Cardano in 12 hours

Victor Isaac — Wed, 21 Oct 2026 20:17:11 +0000

Built: 12 Jul 2026, in 12 hours at IndiaCodeX'26 (a Cardano hackathon), Team Pragmatic. Result: did not place. Status: finished, kept as my first build. Live: exam-guard-omega.vercel.app

This was my first hackathon build. It didn't win anything, and I learned more from it than from some that did, so it gets a post.

The idea

Exam papers leak and get doctored. A student or an invigilator holding a PDF has no quick way to tell the official paper from an edited copy.

Exam Guard, which I built with Manasa and Samhika, does one thing: it turns the official paper into a SHA-256 fingerprint and writes that fingerprint to the Cardano blockchain. Later, anyone can upload a copy. Exam Guard hashes it again and compares. Same hash, authentic. Different hash, changed.

The whole thing is small

Hashing, in Python:

def hash_file(filepath):
    with open(filepath, "rb") as f:
        return hashlib.sha256(f.read()).hexdigest()

For our sample paper, "Physics Mid-Sem 2026", that gives:

ee47d1dab3dd41ada1687923306bd864523bf1ac53269fb681e183e6ddc3c581

That is the number printed as filled bubbles on the cover of this post, one column per character.

Writing it to the chain, in Node with MeshSDK and Blockfrost on the preprod testnet. The fingerprint rides as transaction metadata under label 674, the label Cardano uses for transaction messages:

const tx = new Transaction({ initiator: wallet });
tx.sendLovelace({ address: await wallet.getChangeAddress() }, "1000000");
tx.setMetadata(674, { fingerprint, exam: record.exam_name });
const txHash = await wallet.submitTx(await wallet.signTx(await tx.build()));

The transaction sends 1 ADA back to its own wallet, because the payment is not the point. The metadata is.

What I would tell myself on 12 July

Looking back with a few more builds behind me, the idea has three real problems, and they are more interesting than the code:

A hash proves integrity, not secrecy. A leaked paper is still authentic. Exam Guard can tell you a copy wasn't edited; it can't tell you the paper should never have been in your hands. We pitched it partly as a leak solution, and it isn't one.
It verifies files, not paper. Change a single byte, re-save the PDF, even its metadata, and the hash changes. A printed paper can't be hashed at all. A real version needs a fingerprint of the content, or a signed QR code printed on every page.
The chain is the least important part. Most of the trust comes from who publishes the fingerprint and when. A signed record from the exam board, with a timestamp, does most of what the blockchain does here. The chain adds a public, tamper-evident timestamp, which is useful, but I should have said that plainly instead of leading with it.

None of that is a reason to be embarrassed about the build. It is the honest list I now write for every project, and Exam Guard is where I started writing it.

What I took from it

Ship something that works end to end, even if it is small. Ours hashed, wrote to a real testnet and verified, live, in 12 hours.
Say what it does not do before a judge does. I didn't do that here; every build since has a limits section.

Repo: github.com/victorisaacchinta-eng/exam-guard

I wrote this with AI assistance, working from the repo. The hash above is the real fingerprint in record.json.

Thousands of alerts, one root cause, and an engine allowed to say "I don't know"

Victor Isaac — Wed, 21 Oct 2026 20:16:15 +0000

Built: September 2026, for the Microsoft Codathon (Problem 3), team of five. Result: 1st in our college, 9,900 points. Status: shipped, open source.

When one microservice fails, every service that calls it starts failing too. The on-call engineer gets thousands of alerts, and the loudest one is usually a victim, not the cause.

Problem 3 at the Microsoft Codathon asked for an AI-powered root cause analysis engine. Our team, Ace Azael, built Antibody: it reads logs, metrics, traces and service events, folds everything that belongs together into one incident, and ranks the probable root causes with evidence and a confidence label. I led the technical explanation in our pitch.

The part I am proudest of is one rule we wrote down on day one.

Automation before AI

Every decision in Antibody is deterministic and replayable: detect, correlate, rank, suggest. The language model only rewrites evidence the engine has already computed into plain English, and that text is labelled as AI-generated. If the model is down, nothing breaks; a template writes the explanation from the same evidence.

That rule shaped everything else. Here is the pipeline:

ingest     learn the dependency graph from trace spans (caller -> callee)
detect     per-service EWMA baselines, z-scores, error-log and error-span ratios
           debounce: 2 bad ticks in a row before anything counts
correlate  a new abnormal service joins an open incident if it calls into the failure
rank       8 signals -> an RCA score from 0 to 1 (not a probability)
verdict    HIGH / MEDIUM / LOW from fixed rules. LOW means "insufficient evidence"
suggest    fault signature -> restart, roll back or scale out
human      acknowledge -> apply fix -> verify recovery -> resolve -> remember

The eight ranking signals include personalised PageRank on the anomaly graph, which suspect went wrong first, how many of the callers' error logs blame it, a local fault signature (crash, bad deploy, leak, CPU, slow queries), a recent deploy, and a match with a past resolved incident.

An engine that is allowed to say "I don't know"

Most demos rank something first and present it with total confidence. We wanted the confidence label to mean something, so it comes from fixed rules: how far the top suspect leads the second, how many independent kinds of evidence agree, and whether anything contradicts it. When the rules aren't satisfied, Antibody abstains.

The benchmark, including where it loses

84 scenarios: 7 services, 6 fault types, 2 seeds each, every one with a known cause. The engine never sees the ground truth; it is only used for scoring. python bench.py reproduces it in about 7 seconds.

Root cause ranked first	Clean	Hard mode (random red-herring blips)
Loudest anomaly only	88.1%	84.5%
Anomaly + earliest onset	100%	75.0%
Onset + dependency graph	100%	79.8%
Antibody	100%	84.5% (top 3: 92.9%)

On the confidence labels in hard mode: HIGH is given 78.6% of the time and is right 98.5% of the time. LOW, the abstain, is given 8.3% of the time, and a forced guess there would have been right only 42.9% of the time. When Antibody does answer, it is right 95.8% of the time.

Now the part a judge would find anyway, so we put it in the README first:

On clean traffic, two simpler baselines also hit 100%. Our simulator is too easy to separate the models there.
In hard mode the full model only ties "loudest anomaly" overall. It wins every quiet-crash cascade, where the root goes silent while its callers get loud. It loses when the fault is high in the graph and a red herring sits below it.
The confidence label flags 5 of those 6 losses as LOW or MEDIUM. So when it is wrong, it mostly knows it is unsure.

Other numbers: two unrelated faults at once were found as separate incidents in 6 of 6 pairs, median time to detect is 2 seconds, and a database crash folds about 2,000 events into one incident with 5 services alerting and db ranked first.

What it doesn't do yet

The 7 services are simulated. The ingest API accepts real data; we have not connected a real system.
Fault signatures are pattern rules, so a new kind of failure gets "investigate", not a confident fix.
Memory of past fixes is in-process and resets on restart.
No auth on the API. It is a local demo.

The lesson I took from Antibody, and have reused in every build since: decide in code, explain with the model, and make "I'm not sure" a first-class answer.

Repo (MIT), with the benchmark and tests: github.com/victorisaacchinta-eng/antibody

I wrote this with AI assistance, working from the repo and its benchmark. Every number above comes from bench.py and the README.

Take machines apart with your bare hands: building ATLAS Live in public

Victor Isaac — Wed, 21 Oct 2026 20:16:14 +0000

Status: building now. Started 21 Oct 2026 for the Nebius x NVIDIA Global AI Hackathon. Submission due 30 Oct 2026. Live build: atlas-live-xi.vercel.app

A few weeks ago I built a gesture interface for maintenance technicians, JAMES. One part of it, called ATLAS, drew the targeted machine as an exploded schematic you could pull apart with an open palm. For this hackathon I am rebuilding that part on its own, in the browser, and taking it much further.

ATLAS Live turns real machines, real objects and real places into holograms you can pull apart with your hands, pick with a pinch and question out loud. Every fact it speaks comes from a web search it cites.

This post is written mid-build, so it is a progress report, not a victory lap.

What it does today

Mode	You do	It does
Library	Pick a machine: Perseverance, Ingenuity, the James Webb telescope, a chronograph, a concept car	Loads the real model. Open palm explodes it, pinch picks a part, ask anything
Scan	Hold an object up to the webcam, or drop in a photo	Nemotron vision names it and boxes its parts. Depth Anything V2 lifts the photo into a 3D point cloud, split into those parts
Places	Name a landmark	Google Photorealistic 3D Tiles fly you in. Cited captions for what you are looking at

The models come from NASA (public domain) and the Khronos sample set (CC BY 4.0), with textures and logos removed.

The stack, as an exploded view

The cover of this post is the app taken apart, layer by layer, because that is honestly how I think about it:

Hands: MediaPipe Hand Landmarker, running locally. Camera frames never leave the browser except the single frame you choose to scan.
Eyes: NVIDIA Nemotron 3.5 Super VL identifies a scanned object and locates its parts. A Nemotron agent answers questions and drives the hologram with tool calls (web_search, focus_part, set_explode).
Depth: Depth Anything V2 Small via transformers.js, also in the browser.
Facts: Tavily search. Captions and answers carry their sources.
Ground: Google Photorealistic 3D Tiles through Cesium, on three.js.

All model calls run server-side in Vercel functions. Keys never reach the browser.

Three things that bit me in the first two days

1. Model ids die

The provider layer is any OpenAI-compatible endpoint, first key wins: Nebius Token Factory, then the NVIDIA API catalog, then fallbacks. Right now the live site runs Nemotron through NVIDIA's API catalog, and it will switch to Token Factory the moment that key is set, without a code change.

What I didn't expect: model ids get retired under you. Gemini 2.5 shut down on 16 Oct 2026, days before I started. So the app no longer trusts a hard-coded id. On start it reads the provider's live /models list, and if the configured id is gone it takes the first live id that matches a preference list:

const NEMOTRON_PREFER = {
  vision: [/nemotron.*vl/i, /vl/i],
  agent:  [/nemotron.*super/i, /nemotron.*ultra/i, /nemotron/i],
  fast:   [/nemotron.*lightning/i, /nemotron.*nano/i, /nemotron/i],
};

A model that returns 429 is skipped for 15 minutes and the next one takes over.

2. One image per request

Build mode researches a landmark, plans it, has small builder agents make the parts, and then a critic compares the result with a reference photo. The natural request sends both images. NVIDIA's Nemotron VL endpoint rejects a second image in one request.

The fix was boring and it works: stitch the render and the reference into one side-by-side image and send that.

3. Some famous places look bad in 3D Tiles

Google's 3D coverage is excellent for the Colosseum and Machu Picchu and poor for others. The Taj Mahal looks melted. So I am adding photo reconstructions: a Colab notebook pulls openly licensed photos of the landmark from Wikimedia Commons, matches them (SuperPoint and LightGlue through hloc, then COLMAP), and trains a Gaussian splat with gsplat. The app shows it with the Spark viewer.

One bug worth sharing from that work. In the gsplat version I am using, the opacity reset never fires:

if step % reset_every == 0 & step > 0:

In Python & binds tighter than ==, so this reads as step % reset_every == (0 & step) > 0, a chained comparison that ends in 0 > 0. It is always false. My training loop does the check itself.

Still to do before 30 Oct

Run the reconstruction notebook for the Taj Mahal and Charminar and wire the results in
Test hand control on a real webcam, in poor light
Re-test Build mode on the Eiffel Tower and the Taj Mahal
A 3-minute video and the write-up of where Nemotron and Token Factory are used

If you have tried hand tracking in the browser and found a pinch detector that survives bad lighting, I would love to hear how you did it.

Repo (MIT): github.com/victorisaacchinta-eng/atlas-live

I wrote this with AI assistance, working from the repo and my build notes. It describes the build as of 22 Oct 2026.

A safety gate with no AI in it: building JAMES, a touch-free maintenance copilot

Victor Isaac — Wed, 21 Oct 2026 19:59:17 +0000

Built: first version over 25 to 26 Sep 2026 for AgentX-2026, public on GitHub since 26 Sep.
Status: still being built. ATLAS, CANTUS, SIGHT and the hash-chained ledger landed through October; next is a tuning session on real hands.

A technician with gloves on and both hands busy cannot scroll a PDF manual.

That one sentence is the whole reason JAMES exists. The fix for a vibrating pump lives in four places at once: the manual, the machine's sensor log, the notes from the last time someone repaired it, and the head of whoever did that repair. The technician is standing in front of the pump, holding a spanner, and every one of those sources assumes a free hand and a screen.

So I built a computer you operate by pointing at the air.

JAMES (Joint Autonomous Multimodal Execution System) watches a webcam, listens to a microphone, finds the right procedure, checks the sensor log and past repairs, and walks the technician through the fix one step at a time. Point at the machine, ask out loud, pinch to confirm each step. It runs entirely on a MacBook Air: local model, local search, local memory, cloud off.

It took first prize at AgentX-2026, a national-level hackathon at TKREC Hyderabad. But the part I want to write about is not the demo. It is the three ideas that made it safe to put in front of a jury, and the ugly bugs that taught me each one.

Idea 1: the model can suggest a step. Only plain code can allow it.

JAMES is a small team of agents, each with one job:

Agent	Job	Built with
LENS	Sees hands and the machine's asset tag	MediaPipe Hand Landmarker, OpenCV ArUco
ECHO	Hears the request	faster-whisper, Qwen3-VL 4B via Ollama
PAGE	Finds the procedure and cites the page	PyMuPDF, BM25, nomic-embed-text in Qdrant
PULSE	Reads the sensor log	pandas, and it never invents a reading
LEDGER	Remembers approved jobs	append-only SQLite
FOREMAN	Plans and decides the likely cause	LangGraph fan-out to PAGE, PULSE, LEDGER
WARDEN	Blocks unsafe steps	plain Python and a rule file. No AI.

WARDEN is the important one. The model is allowed to be creative about what to check next. It is never allowed to decide whether a step is safe. That decision lives in a rule engine anyone can read, with rules that ship in an "industry pack" next to the manual and the asset tags.

In the demo, step 4 of the pump repair is "Remove the coupling guard". If the technician skips lockout, WARDEN refuses:

BLOCKED BY SAFETY POLICY   LOTO-01, ZERO-01
Remove the coupling guard
Lockout and zero energy are not confirmed.

Two pinches later (lockout confirmed, zero energy confirmed), WARDEN re-checks and the step opens. The seeded safety suite runs 8 known violations and 4 legitimate sequences: 8 of 8 violations blocked, 0 of 4 false blocks.

The pack is hash-checked too. If anyone edits a rule file after its manifest was written, the app refuses to start. A safety rule you can quietly change is not a safety rule.

Idea 2: the pinch has to be a pinch

The first live run on the Mac taught me that hand tracking fails quietly, and that the failures look exactly like intent.

A fist reads as a pinch. In a fist the thumb lies on the index finger, so the thumb-to-index distance drops under the pinch threshold. Closing your hand to rest it was confirming steps.
Reaching for a tool reads as a pinch. Hold a pinch, move your hand toward the camera to grab a spanner, and you have "confirmed" whatever was on screen.
Losing the hand was silent. In low light or with dark gloves the tracker just stopped, and the app kept showing the last known state as if nothing happened.

None of these were tuning problems. They were design problems, so each got a design fix:

Fist latch. A pinch that begins as a fist is ignored until the fingers open again. The test for "this is a pinch, not a fist" is that the index finger is at least half extended.
Reach guard. While a pinch is held, JAMES tracks hand size (the geometric mean of palm length and knuckle width, so tilting moves it about half as much as reaching). If it changes more than 30%, the hand is moving to or from the camera. The pinch is dropped and must be released before it counts again. A reach never confirms a step.
SIGHT. A small module that watches frame brightness and how often the hand appears and vanishes. When tracking can't be trusted it says so on screen, and a lost hand is marked with a still amber X where it was last seen, the way cockpit displays mark invalid data. SIGHT can warn. It can never press or unlock anything.

The research backed this up. One study of 52 person-hours of everyday activity found a naive gesture spotter fired a false positive about every 5.1 seconds. That number changed how I thought about every gesture in the app: the default assumption is "this was an accident", and the gesture has to prove otherwise.

Idea 3: every answer has a source, and every record can be checked later

A maintenance copilot that makes things up is worse than no copilot. So:

Steps come from real numbered lines in the manual, word for word, each with its page.
Readings come only from the supplied sensor log. PULSE never estimates a value.
Past repairs are recalled only if a senior approved them, they were not revoked, and they are for the same class of equipment. Memory can suggest. It never decides a cause on its own.
The status bar always says which backend is actually live (local model or keyword rules, BM25 or BM25 plus Qdrant), so the demo can't overstate itself.

And because buyers in regulated plants ask whether a record has been altered, every logged event is hash-chained: each event stores SHA-256 of the previous hash plus the event itself. A read-only checker walks the chain and names the first edited, reordered or removed event. It is tamper-evident, not tamper-proof, and the README says exactly that.

How it was built

In short iterations with the same loop every time:

Review the build and list findings as P0 (must fix before a live demo) or P1 (should fix).
Fix each one and add a regression test that fails without the fix.
Run it live on the Mac. Every sentence that misbehaved in the app log became a new test, and the whole log was replayed through old and new code.

Some of my favourite bugs from that log:

Playing the neural voice through PortAudio in the same process as the microphone crashed on macOS. Each sentence now plays through afplay in its own process.
A model plan searched a website the user never named. Search and collect steps must now name a site the user actually said.
Gesture timers reset when the hand moved at all, so "explode the model" felt dead, then fired all at once.

The repo has 607 tests passing, and the build log marks every claim as LIVE (seen on the Mac), TESTED (automated) or UNVERIFIED. That labelling is the most useful habit I picked up on this project. It stops me quoting a number I never actually saw.

Then I made it fun

Two side apps grew out of the hand tracking:

ATLAS draws the targeted machine as an exploded line drawing and lights the part the current step touches: amber when WARDEN allows it, red when WARDEN is blocking it. It also runs as its own app with a library of openly licensed models (NASA's Perseverance rover, Ingenuity, the James Webb Space Telescope) that you take apart by opening your hand.
CANTUS is a rhythm game in the launcher. Blocks fly towards you on the beat; swing through them, left hand red, right hand blue.

A pinch inside ATLAS can never confirm a maintenance step. ATLAS owns the hand while it is open. Fun is allowed. Ambiguity about what a gesture means is not.

What is not done

It ships with sample data and demo safety rules. Nothing here is certified for real safety use.
Industry packs are hash-checked but not signed yet.
It runs on macOS only today.
SIGHT thresholds, the reach tolerance and the gesture timings still need a proper tuning session on the Mac before I quote them as final.

Try it

The code is public under MIT: github.com/victorisaacchinta-eng/james. The product site is jamesagent.vercel.app. python app.py --sim runs a full rehearsal with no camera or microphone.

If you have built gesture interfaces and have opinions about false activations, I would genuinely like to hear them in the comments.

I wrote this with AI assistance, working from the repo and its build log. Every number above comes from the repo.

Code decides, AI explains, a named officer signs: bid compliance for Smart India Hackathon

Victor Isaac — Wed, 21 Oct 2026 19:59:11 +0000

Built: late September 2026 for Smart India Hackathon 2026 (repo public since 29 Sep).
Status: submitted; the live workstation stays up for anyone to try.

A procurement officer checking a government bid in India opens portal after portal. GST. Income tax. MCA21 for the company record. Udyam for MSME status. EPFO and ESIC for labour compliance. The debarment list. Then the bid documents themselves, to see whether the name on the PDF matches the name on all of the above.

It is slow. Worse, two officers can look at the same bid and reach two different answers.

For Smart India Hackathon 2026 (problem statement SIH26100, from the Ministry of Petroleum and Natural Gas), our team Ace Azael built Procurement Pilot: a workstation that runs fifteen fixed checks on every bid against ten registry sources, scores it out of 100, explains every flag in plain language, and then hands the decision to a named officer.

The design rule fits on one line:

Code decides. AI explains. A named officer signs.

This post is about why that order matters, and what it looks like in code.

Why the model doesn't get to score

The tempting build is obvious: give an LLM the bid, the tender and the registry data, and ask it whether the bid is compliant. It would demo well. It would also fail the one property a government process needs most: the same bid must get the same answer every time, and an auditor must be able to see why.

So every check is plain Python. Each one returns PASS, FLAG or FAIL, plus a sentence naming the real numbers and the source it checked. The score is arithmetic over those results:

def score_and_verdict(checks):
    raw = sum(c["points"] for c in checks)       # PASS = full weight, FLAG = half, FAIL = 0
    fails = [c for c in checks if c["status"] == "FAIL"]
    hard = [c["name"] for c in fails if c["name"] in HARD_CAP_CHECKS]
    cap = 45 if hard else 72 if fails else None   # a hard FAIL caps the score at 45
    value = min(raw, cap) if cap is not None else raw
    score = int(value + 0.5)
    ...

The weights add to 100. A FAIL on any of four hard checks (EMD, statutory documents, GST registration, debarment) caps the score at 45 and makes the risk Critical, no matter how good the rest of the bid looks. Any other FAIL caps it at 72. The officer sees a "Why not 100" line listing every point lost and why.

That cap was the most important design decision in the project. Without it, a debarred vendor with perfect paperwork everywhere else could score 92. With it, debarment means the bid is Non-compliant.

The rules that need no registry are real

We could not get live access to government registries in a hackathon. That needs onboarding through API Setu and a GST Suvidha Provider. So every registry lookup reads sample records, and the app labels every one of them as simulated on screen.

But some checks need no lookup at all, and those are real. A GSTIN carries its own check digit:

def gstin_check_digit(first14: str) -> str:
    total = 0
    for i, ch in enumerate(first14):
        product = _CHARS.index(ch) * (1 if i % 2 == 0 else 2)
        total += product // 36 + product % 36
    return _CHARS[(36 - total % 36) % 36]

A GSTIN that fails this test cannot be a valid registration, whatever the bid claims. The PAN sits inside the GSTIN, and its fourth character tells you the holder type, so a "Private Limited" company filing with an individual's PAN gets flagged. Make in India class thresholds (Class-I at 50% local content, Class-II at 20%), EMD exemptions for verified MSE, Startup and NSIC bidders, and incorporation under 91 days as a shell-company signal are all real rules too.

Being honest about what is simulated is not a weakness in the pitch. It is the pitch. A judge who finds an unlabelled fake will distrust everything else.

Where the AI actually earns its place

The reasoning layer (Sarvam's sarvam-105b, an Indian model) does one job: it reads the check results and writes the explanation an officer would want to read. It has three hard limits:

It cannot set the score or the verdict. Those are computed before it is called.
It never sees a full PAN or GSTIN. Identifiers are masked before anything reaches the model:

   def mask_ids(text: str) -> str:
       """The model never sees a full PAN or GSTIN: the PAN part is masked in both."""

It is optional. Without an API key, the app writes a deterministic summary from the same evidence and says so. The demo never depends on Wi-Fi.

Catching what a single bid can't show

Some fraud only appears across bids. For every pair of bids on a tender, Procurement Pilot screens for:

prices within 0.5% of each other
submissions within 5 minutes of each other
shared addresses
cover bids more than 25% above the benchmark

In our demo data, two transport bidders quoted 0.38% apart and submitted 2 minutes apart. One of them had been incorporated 55 days before bidding and had no ESIC registration. No single-bid check would have connected those facts.

A human signs, on the record

The last step is deliberately not automated. Officer, senior approver and admin roles are enforced by the server, not the UI. Passing a flagged bid needs a senior approver and a written reason of at least 30 characters. Every decision, with name and time, lands in an audit log that an admin can export.

The stack

FastAPI running as a Vercel Function, Postgres on Neon (SQLite locally)
JWT sign-in with server-enforced roles
A PDF reader that pulls GSTIN, PAN, Udyam, CIN, DPIIT number and legal name from a bid document and cross-checks each one
26 API tests, plus a parity test that runs the frontend's offline engine against the backend for every sample bid. CI runs both on every push

What I'd tell another team

If your agent touches money, eligibility or safety, split it in two. Let deterministic code own every decision that must be repeatable, and let the model own the part humans are bad at: turning fifteen results into two clear paragraphs. You get an AI feature judges can see, and a system an auditor can trust.

The live build is at procurement-pilot-sih.vercel.app and the code is public under MIT at github.com/victorisaacchinta-eng/procurement-pilot-sih. There is also a 4:52 walkthrough on YouTube.

Student prototype. Not an official government system, and every tender, bid and registry record in it is sample data.

I wrote this with AI assistance, working from the repo. Every number and code excerpt above comes from the repo.

Every bank statement carries its own answer key

Victor Isaac — Wed, 21 Oct 2026 19:59:05 +0000

Built: 18 to 21 Oct 2026.
Status: live at ithamar.vercel.app. Next up: real statements from specific banks, then scanned PDFs.

Bank statement extractors get rows wrong quietly. A 7 read as a 1, a debit dropped into the credit column, one row lost at a page break. Nobody notices until the totals stop adding up, and by then the bad numbers are already in a spreadsheet, a loan file or a tax return.

The thing is, every bank statement already carries its own answer key. Each row obeys one line of arithmetic:

balance above + credit - debit = balance printed

So I built Ithamar, which reads an Indian bank statement PDF, walks that equation down the page, and marks every row as one of three things:

Mark	Meaning
✓ proven	the arithmetic holds exactly as printed
↻ repaired	it holds after one correction, and the correction is logged beside the row
✗ left for you	no correction is earned; Ithamar says what it found and stops

It never guesses, and it never invents a transaction.

Parsing without templates

Most extractors keep a template per bank. I wanted one parser that learns each statement's layout from the page itself.

parse/layout.py finds the header row, learns where each column sits from its header words, then assigns every word below it by geometry: amounts align to a column's right edge, text to its left. A line without a date continues the row above, which is how wrapped narrations survive. It handles lakh grouping (1,00,000.00), Dr/Cr suffixes, single amount columns with a separate direction column, and headers repeated on every page.

And one rule that saved me hours of debugging later: all money is integer paise. Floats never touch an amount.

When a row breaks

If the equation fails, the naive move is to stop, or to "fix" the balance so everything below lines up again. Both are wrong. Stopping hides every later row. Forcing the balance hides the actual error.

Instead, the chain resyncs on the printed balance, so one bad cell makes one or two breaks rather than corrupting everything below it. Then each break is tested against four concrete explanations, each with the exact correction it implies:

The amount is in the wrong column.
A digit in the amount was misread: one substitution, drop, insertion or adjacent swap, or a decimal shift.
A digit in the balance was misread. Its row breaks, and so does the row below.
A whole row is missing.

Repair is a decision, not a guess

This is the part I am proudest of. A correction is applied only when both of these hold:

it is the only specific explanation that closes the local window: the broken row and the next row that prints a balance must both agree afterwards
it strictly reduces the number of breaks

And a missing row is never applied. "There was a transaction here that I can't see" explains any break, which means it proves nothing. So Ithamar flags it for a person and leaves it alone.

There is one more subtle rule. A row the engine corrects ends as repaired, never proven. "Proven after a change" is a different claim from "proven as printed", and the output should never blur the two.

The metric that matters

A benchmark that rewards fixing things will happily reward wrong fixes. So the headline number is the opposite: false-verified rows, rows marked proven that are actually wrong. It must stay at zero.

ithamar bench renders statements whose truth is known by construction, injects extraction-style errors into what is printed (never into the truth), and scores the engine:

24 statements, 855 rows, 39 injected errors
every swap, amount misread and balance misread repaired to its true value
every missing row flagged and left alone
false-verified rows across 24 statements: 0

To be clear about what that shows: these are synthetic statements from the project's own generator. They show the verifier works. They do not show that the parser handles every real bank. Real statements are the next test, and I will publish those results the same way, failures included.

The test suite leans on property tests with Hypothesis. The headline property is that whatever single error is injected, across hundreds of generated ledgers, no row is ever marked proven or repaired unless its values equal the truth.

Two engines, one behaviour

The command line runs a Python engine on pdfplumber. The website runs a TypeScript port on pdf.js, so a statement is parsed and proven entirely inside your browser tab. Nothing is uploaded, and there is no server.

Two engines invite drift, so a parity script runs both on the same PDFs and fails on any difference in rows, amounts, statuses, repairs, unresolved rows or decoded narration fields. CI runs it on every push.

Each audited statement also gets a seal: a guilloche rosette, like the security printing on a cheque, drawn from that statement's own balances. Every row owns a slice of it. Proven rows draw clean, repaired rows draw in violet, and a row that does not balance prints out of register in magenta, exactly where it broke. It is the most fun I have had making a correctness check visible.

What the chain proves, and what it doesn't

Proven by arithmetic	Not proven
Debit, credit and balance of every verified row	Dates
No transaction is missing between two verified rows	Narration text and counterparty names
The closing balance matches the last row	Rows on statements that print no running balance

Narrations are still decoded (UPI, NEFT, IMPS, RTGS, VPA, RRN, UTR, IFSC and a best-effort counterparty), and the output says plainly that none of that is proven by the chain.

Why the name

Exodus 38:21 records the first audit: the materials of the tabernacle were counted "by the hand of Ithamar". And its numbers still check. Exodus 38:25 to 26 counts 603,550 men giving half a shekel each, which is 301,775 shekels. It separately reports the total as 100 talents and 1,775 shekels, which at 3,000 shekels to the talent is also 301,775. Two independent records, one total. That is exactly what this tool does to a bank statement.

Try it

In the browser, nothing uploaded: ithamar.vercel.app
The code, MIT: github.com/victorisaacchinta-eng/ithamar

pip install git+https://github.com/victorisaacchinta-eng/ithamar
ithamar demo        # generate synthetic statements and audit them

ithamar audit exits 0 when every row is proven, 1 when rows were repaired, and 2 when a person needs to look, so it can gate a script.

If you have run into a statement layout that breaks parsers, tell me in the comments. That is exactly the test data I need next.

I wrote this with AI assistance, working from the repo. Every number above comes from the repo's benchmark.

My browser agent took 6 tries on Blinkit and 1 on a site it had never seen

Victor Isaac — Wed, 21 Oct 2026 19:58:58 +0000

Built: 12 Sep 2026, solo, in one day at the SLAB Hackathon.
Status: finished hackathon build. The repo has a prototype of the cached version described at the end.

My browser agent needed 6 attempts to get a clean price out of Blinkit. A few hours later it got one out of District, a site it had never touched before, on the first try, live, on stage.

The agent didn't get smarter in between. It got a notebook.

This is Basket Buddy, which I built solo in a day for the SLAB Hackathon (Self-Learning Agent Browser) at CMR Institute of Technology, Hyderabad, Track 05: Self-Learning Loops. It took second place. The idea is small, and I think it generalises to a lot of agent work: make the agent write down what it learned, in a form it can read back next time.

The task

Search a quick-commerce site for a product ("milk"), and extract the name, size, price and MRP of the results. Do it across several platforms so someone can compare a basket without opening three apps.

Every browser action ran through WebCMD sessions, one throwaway profile and session per platform. No raw Playwright or Selenium script drives anything directly.

Attempt counts, not vibes

Here is the actual log, attempts before a working, clean extraction:

Platform	Attempts	What happened
Blinkit	6	Cold start. No strategy existed yet
BigBasket	2	Blocked. A real 404 from the site itself, likely bot detection. Correctly abandoned
Amazon	1	Reused a lesson from Blinkit
Zepto	2	URL worked first try; one fix for discount-badge noise
District	1	Transfer test. Zero prior exposure. Live

The number I cared about was never "did it work". It was "did it need fewer tries because of what it learned before".

What Blinkit taught it

Blinkit was painful, and every bit of pain became a line in a file. This is the real strategy the agent wrote after cracking it:

{
  "platform": "blinkit",
  "search_url_pattern": "https://blinkit.com/s/?q={query}",
  "product_card_selector_note": "No stable class names; find ADD buttons with no children, then walk up 3 parents to reach the card",
  "extraction_note": "Parse price via regex on innerText (₹\\d+), not by selector; first ₹ match is price, second is MRP if discounted",
  "gotchas": ["Homepage search box is a fake trigger link (href=/s/), not a real input, don't try to click+type it"]
}

Look at that gotchas line. The search box on the homepage looks like an input, but it is a link that opens the real search page. An agent that clicks and types into it fails in a confusing way, every time, forever, unless it remembers.

The lesson that transferred was not "Blinkit's selectors". It was more general: verify the direct search URL before touching the DOM. Amazon worked first try because of it. Zepto's URL worked first try because of it. District, a site with no strategy file at all, worked first try because the agent reached for the URL pattern first instead of poking at a search box.

Two kinds of memory

There are two separate memory layers here, and keeping them apart is what made this clean:

Site structure memory belongs to WebCMD itself: explore a site once, reuse the map.
Task memory is my layer on top: strategies/*.json, one file per platform, written after each run with the search URL pattern, selector notes and gotchas. What worked, what didn't, and why.

The loop is simple:

plan the run
  -> site known?  no  -> explore cold, learn, write strategies/<site>.json
                  yes -> reuse the strategy
  -> extract name, size, price, mrp
  -> next run starts from what this run wrote

Knowing when to stop

BigBasket returned a consistent real 404 to the automated browser. The agent logged it and moved on.

That sounds like a failure in the table, but I think it is one of the most important rows. An agent that "learns" by retrying until something gets through is not learning, it is brute-forcing someone else's site. Abandoning a blocked platform is the correct behaviour, and it should be recorded as a result, not hidden.

The other guardrails were enforced in code, not described in a prompt:

Every run stops at read-only extraction. No checkout, no payment, no account action.
Human confirmation is required before anything irreversible.
Throwaway profiles only. No cookies or credentials are committed to the repo.

What I'd change for a real product

Live scraping per visitor doesn't scale, and it would invite more BigBasket-style blocks on every platform. The repo has a prototype of the better pattern in server/: a background job is the only place the agent runs, on a schedule, and an Express server serves the cached results instantly.

The hard parts left are not code problems: prices vary by pincode because dark stores do, a real product should never ask a visitor for their Blinkit password, and scraping at scale raises terms-of-service questions that more engineering doesn't answer.

The takeaway

If your agent keeps making the same mistake on the same website, the fix is often not a better model or a longer prompt. It is a small, human-readable file the agent writes after each run and reads before the next one. You can open it, check it, and correct it. And when the agent finally cracks something, the lesson is still there tomorrow.

The repo, with the full learning log and every strategy file: github.com/victorisaacchinta-eng/basket-buddy

I wrote this with AI assistance, working from the repo and its learning log. Every attempt count above comes from that log.

My Agent Kept Forgetting Our Company, So I Gave It Hindsight

Victor Isaac — Mon, 28 Sep 2026 22:01:47 +0000

In February, a lender sent my father's interior fit-out company a vendor pre-qualification questionnaire. Forty-odd questions, a folder of supporting documents, and a deadline of the next afternoon. The company was qualified. It had already built six gold-loan branches for another lender. Five months and two reminder emails later, the questionnaire was still not submitted.

Nobody was lazy. The answers existed. They were just scattered across old email threads, previous submissions and one person's head. Every new questionnaire started from zero.

That is the problem we built Prequal to solve: an agent that answers vendor questionnaires from a permanent memory of one company, shows the evidence behind every answer, and gets better every time someone corrects it.

Why a normal LLM agent fails here

Our first instinct was the obvious one: paste the company profile into a prompt and let a model fill in the questionnaire. It fails in three ways.

It forgets. Each questionnaire is a fresh session, so last month's corrections are gone.
It guesses. Ask a model "Do you hold ISO 9001?" with thin context and it will happily produce a confident, plausible, wrong answer.
It can't show its work. A reviewer has no idea which fact an answer came from.

For vendor paperwork, the second failure is the dangerous one. A fabricated certification on a pre-qualification form is worse than no answer at all.

So the design goal became: the agent should only say what it can prove, and its memory should outlive any single session. That's where Hindsight agent memory came in.

How the system hangs together

Prequal is a small FastAPI backend, a single-page review screen, SQLite for questionnaire state, Groq for the language model, and Hindsight as the only place company knowledge lives.

Every question goes through the same pipeline:

Classify. Each question has a type: exact field, document request, yes/no, numeric by year, or free text.
Deterministic answers first. GSTIN, PAN, entity type, address: these are looked up directly. No model call.
Document check. "Attach your ISO 9001 certificate" is checked against the company's document inventory. On file, or not held. Still no model.
Recall. Everything else pulls evidence from Hindsight.
Draft with tools. The model must call submit_answer with the evidence IDs it used, or flag_gap. Plain text is never treated as an answer.
Guardrails. Low confidence or uncited drafts go to human review. Any positive claim about a certification the company doesn't hold is blocked.

Automation before AI. The model only sees the questions that actually need judgment.

The core story: memory is the product

Here is the part that surprised us. Without memory, Prequal is useless. With an empty memory bank, every single question comes back as "needs review". The agent has nothing to stand on, so it says so.

That's exactly the behaviour we wanted, and it's only possible because the agent treats Hindsight as its source of truth rather than as a nice-to-have.

We give each company its own memory bank with a mission and three directives:

DIRECTIVES = [
    ("no-fabrication", "Never state a certification, registration, licence or figure that is not in memory. If it is absent, say it is absent.", 1),
    ("timeline", "When a fact has more than one value over time, report the timeline, most recent first, and name the dates.", 2),
    ("reviewer-wins", "Prefer the reviewer's corrected answers over the agent's own earlier drafts.", 3),
]

On day one we retain three kinds of memory: company facts, one fact per document (on file or not held), and every answer the company has given on past questionnaires. Turnover is retained once per financial year, with a timestamp:

for fy, v in p["turnover_by_fy"].items():
    w(f"{name}'s annual turnover for FY {fy} was {v['label']} (audited).",
      context="annual turnover from audited accounts",
      ts=_fy_end(fy), tags=("profile", "financial"))

That timestamp matters. When a questionnaire asks for "turnover for the last three years", recall returns dated facts, and the review screen renders them as a timeline, newest first. When a new year's figure is retained, it simply appears at the top. No overwriting, no stale numbers.

The learning loop

The part engineers ask about most is how the agent improves. It's almost boring:

def retain_review(question, final_text, reason):
    content = (f"Reviewer-approved answer on the {question['client']} questionnaire "
               f"to the question \"{question['text']}\": {final_text}")
    if reason:
        content += f" Reviewer's note: {reason}"
    m.retain(content, context=f"reviewer correction for {question['client']}",
             timestamp=datetime.utcnow(), tags=["review", ...])

Every approval and every edit becomes a memory, with the reviewer's reason attached. On the next questionnaire, a similar question recalls the corrected answer and the model cites it. If a reviewer says "always lead with the six-branch project", that preference is in memory for every future questionnaire, from every client.

This is the difference between an agent with a longer prompt and an agent with memory. A prompt resets. Memory accumulates.

What it looks like in use

A reviewer opens a new questionnaire and clicks run. Questions stream in with one of three statuses:

Auto-filled: answered from an exact field, a document on file, or cited evidence.
Needs you: thin evidence or low confidence.
Gap: nothing in memory answers it, or the document isn't held.

Click any answer and the evidence panel shows exactly which memories it came from, with the cited ones marked. A live log on the right shows every retain, recall and reflect call as it happens. Before answering, one reflect call produces a short brief: which past projects matter for this client, which documents are missing.

The moment that sold us on the approach: asked "Do you hold ISO 9001 certification?", the agent answers "No. We do not hold an ISO 9001 certificate at present" and marks it as a gap for the reviewer to decide. It doesn't invent a certificate number. It tells you what's missing.

On a full run against a 43-question questionnaire from an NBFC, with real Hindsight memory and Groq's gpt-oss-120b drafting, Prequal answered 38 questions, sent 1 to a human for review, and flagged 4 as gaps. All four gaps were documents the company does not hold. The company in the demo is fictional, modelled on a real firm's structure with synthetic figures.

Here's a two-minute walkthrough:

Lessons learned

1. Deterministic first, model second. In our 43-question sample, 15 questions are exact fields and 13 are document checks. Sending those to a model adds cost, latency and risk for nothing.

2. Force the model to cite, then check the citations. Tool calls with evidence IDs, validated after the fact, turned "trust me" answers into auditable ones.

3. Timestamps are underrated. Retaining facts with the date they became true made "what changed" a query instead of a data-cleaning job.

4. Make the empty state honest. An agent that says "I don't know this company yet" is more trustworthy than one that fills the silence.

5. Function calling fails. Plan for it. We validate every tool argument, retry once with the schema echoed back, fall back to a second model, and treat anything else as "needs review". A crash is never an answer.

What's next

Parsing Excel and PDF questionnaires directly, writing answers back into the client's own template, and document expiry alerts so a solvency certificate never lapses silently.

The bigger lesson is about what agent memory actually is. It isn't a bigger context window. It's a record that outlives the session, knows when things were true, and gets better every time a human corrects it. For a small company answering the same forty questions over and over, that's the whole product.

Code: https://github.com/victorisaacchinta-eng/prequal
Demo: https://youtu.be/cmx4w6JXkEw