DEV Community

Han Vigo
Han Vigo

Posted on AI-assisted

I let httpx's departed reviewers come back for a day. They had notes.

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission πŸ‘»πŸŒΌ

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 4 β€” Ghost in the Machine.

What I Built

Ancestors lets a repository's past reviewers look at today's pull requests.

Every long-lived open-source project has people who reviewed hundreds of PRs and then moved on. encode/httpx is a good example. Of its 2,690 line comments, 675 were written by florimondmanca, whose last one is from February 2023; 255 by sethmlarson, all in 2019; 124 by yeraydiazdiaz, none after mid-2020. Three of the four most prolific reviewers in the project's history have not left a review comment in years. Their judgement did not leave with them. It is still there, in threads nobody reads any more.

Ancestors ingests that history, distils what the reviewers kept asking for, and then reviews a new diff in their collective voice. Three rules make it honest rather than spooky:

  1. Only the dead may speak. A comment counts as an ancestor only if it is more than twelve months old, and the PR under review is never allowed to testify about itself. (My first version got this wrong and happily "discovered" the reviewers' concerns by reading this week's review thread. Fixed, and the fix is the most important line in the repo.)
  2. No citation, no verdict. Every note must cite a comment id that was actually retrieved for that hunk. If the model cites nothing verifiable, the verdict is downgraded to "the ancestors are silent on this".
  3. Two witnesses or it does not count. A "house rule" in the charter survives only if two different comments back it.

It runs entirely on my desktop: Gemma 3 (4B) writes, EmbeddingGemma retrieves, Mastra holds the agent, memory, vector store and the charter workflow, and Temporal turns a multi-hour pass over a repo's open PRs into something that survives me killing the worker.

DΓ­a de los Muertos was the obvious frame. The dead are allowed to visit for one day, and they may only speak about what they actually saw.

Demo

The ancestors visited PR #3769: three notes, each citing a 2020 review comment on the same file
A report as rendered by the little viewer in out/: every note ends in clickable citations; hunks the ancestors never spoke about are listed as silent.

Temporal timeline: reviewPr attempt 1 (red) dies with the worker, attempt 2 picks the same PR up on a new worker, then the second PR runs
The Temporal UI timeline of the run described below: the red segment is the review that died with worker #1; the retry is the same activity on worker #2.

Repo: https://github.com/vigohan95/ancestors (MIT). Everything below is from real runs on a Ryzen 7 5700G with no GPU.

What the ancestors actually said

I pointed it at three open httpx PRs this morning. Every voice quoted below is from 2020–2024; the cutoff means nothing from the last twelve months can speak, and the PR's own thread is excluded.

PR #3769 β€” "Ensure mounted transports are closed even if main transport raises." The PR wraps transport shutdown in Client.close() / aclose() in try/finally. For three of the four hunks the ancestors came back with the same two voices from August 2020, when this exact method was last redesigned:

πŸ•―οΈ httpx/_client.py βˆ’1267,10 +1267,12 β€” The ancestors are concerned about exception handling and ensuring all transports are closed, even in error scenarios. Comment [c477206186] suggests passing exception information and closing proxies, while [c472953172] recommends handling redundant close calls. The current code uses a try...finally block to guarantee transport closure, but the absence of exception handling is noted.

  • [c477206186] florimondmanca, 2020-08-26, PR #1218 on httpx/_client.py: β€œβ€¦await self.transport.aexit_(exc_type, exc_value, traceback) for proxy in self._proxies.values()…”
  • [c472953172] lovelydinosaur, 2020-08-19, PR #1197 on httpx/_client.py: β€œWe should put self._is_closed = True as the very first line, rather than the…”

That is a real thread. In PR #1197 and #1218 the maintainers argued about exactly this close-ordering problem β€” pass the exception down, set _is_closed first, close every proxy even when one raises. The 2026 PR re-opens the same ground without knowing, and the ancestors hand the reviewer the two comments to read before deciding. (4 hunks, 106 s, 3 notes, 1 silent.)

PR #3767 β€” "Include header name in encoding error message." For the hunk in httpx/_models.py that re-encodes header values, the ancestors pulled obataku's 2020 question about why the code does not just use normalize_header_value, and alexmojaki's 2024 aside about self.text for non-UTF-8 charsets. Useful context for a reviewer; probably not a blocker. The other three hunks (tests, an unrelated signature) came back silent.

PR #3777 β€” "Add real async iterator for ByteStreams." Both hunks: silent. The nearest voice (agronholm, 2025-07, on __aiter__() returning any AsyncIterator) is from this summer, so it is not old enough to be an ancestor, and nothing older applied. I like that it says so. A hosted model would have produced three confident paragraphs here.

The current weakness is visible too: when the ancestors conclude "this is acceptable", Gemma 4B still labels it a note instead of honoured. The verdict schema has an ok state; the small model under-uses it. That is a prompt problem I know how to fix and a model-size problem I can fix with one environment variable.

How I Built It

1. The voices

gh api repos/encode/httpx/pulls/comments returns every line-anchored review comment since 2019: 2,690 of them across 168 files, median 119 characters, 1,182 from 2019 alone. Each substantive comment (β‰₯25 characters; 2,442 of them) is embedded together with where it was said β€” file path, reviewer, the last six lines of the hunk under review β€” using EmbeddingGemma through Ollama, and stored in a LibSQL vector index via @mastra/libsql. A blob-less clone supplies the 1,482-commit log, so the agent can also tell you who last touched a file and why.

Ingest, end to end, on CPU: 448 seconds.

2. The charter β€” a Mastra workflow

Before reviewing anything, a three-step createWorkflow (sample-voices β†’ distil-charter β†’ save-charter) samples comments stratified by year (so 2019's flood does not drown 2024), hands them to Gemma, and asks for the expectations the reviewers kept returning to, each with the comment ids that support it. Then the code applies the two-witness rule and throws away everything else.

Here is the entire charter for httpx after the two-witness rule:

Distilled by Gemma from 44 comments spread across every year of review history; a rule only survives if at least two different comments back it (7 single-witness candidates dropped).

  1. Prefer explicit parameters over **kwargs in public signatures. ([c399643358] [c552568019])

One rule. Gemma proposed eight; seven were one reviewer talking about one line, dressed up as tradition. I would rather hand the review agent one true sentence than eight plausible ones, and the sampled comments still reach the agent through retrieval, where they are attributed to a person and a date instead of to "the project".

3. The review β€” a Mastra agent on a very short leash

The agent never gets "review this PR". For each of the most substantive hunks it gets: the hunk, the last five commits to that file before the cutoff, the charter, and the six closest ancestor comments (comments on the same file get a boost β€” the ancestors literally stood here). It has to answer in a strict JSON shape: note, ok, or silent, plus citations.

The line that keeps it honest:

const cited = (parsed?.citations ?? [])
  .map(c => allowed.get(c.replace(/[\[\]]/g, '')))
  .filter(Boolean);
if (verdict !== 'silent' && cited.length === 0) verdict = 'silent';
Enter fullscreen mode Exit fullscreen mode

allowed is the set of comments that were actually retrieved for this hunk. A verdict that cites nothing from that set is silence, whatever the model wrote. This is what makes a 4B model usable: the retrieval does the remembering, the model does the reading, and it cannot pass off invention as tradition.

Per hunk on this CPU: 25–40 seconds. A four-hunk PR: about two minutes.

4. The haunting β€” Temporal

Reviewing one PR takes minutes; reviewing a repository's open PRs takes an hour. That is exactly the kind of job that dies when a laptop sleeps. The hauntRepo workflow lists open PRs and calls a reviewPr activity per PR; the activity heartbeats after every hunk, the worker runs one activity at a time (maxConcurrentActivityTaskExecutions: 1, because there is one local model), and the retry policy does the rest.

Here is the run I kept (workflow haunt-encode-httpx-1793376447857, two most recent open PRs):

time (UTC) what happened
16:07:27 workflow starts; listOpenPrs returns #3796 and #3785
16:07:28 reviewPr(3796) attempt 1 starts on worker #1; hunks 1–2 reviewed, heartbeat after each
16:07:51 I kill worker #1's node process (taskkill /F /T) mid-hunk
16:09:40 heartbeat timeout (2 min) fires; Temporal schedules attempt 2; worker #2, started at 16:08:08, picks it up and reviews all 6 hunks
16:11:20 attempt 2 completes (98 s of model time); reviewPr(3785) runs on worker #2
16:11:39 workflow completes: 2 PRs reviewed, 3 notes, nothing lost, nothing reviewed twice by the same worker

The activity itself is idempotent (it rewrites the report file), so the retry policy is the whole crash story: no checkpoint code, no queue, no cron. The first version of this demo "worked" because my kill command only killed the npx wrapper and the orphaned node process quietly finished the job; the event history is what caught me β€” the retry attempt never appeared. Durable execution is also a lie detector.

Why open innovation matters for this

  • Review comments on a private repo are the last thing you want to ship to a hosted model. Everything here talks to localhost:11434. Unplug the network after ingest and the review still works (--diff file.patch skips GitHub entirely).
  • It has to be cheap enough to run on every PR, forever. The whole httpx history embeds in seven minutes on a five-year-old desktop CPU; a review costs electricity, not tokens.
  • Swappable. ANCESTORS_MODEL=gemma3:12b and the ancestors get more articulate; no code change. The project's values live in the retrieval, the cutoff and the charter, not in a model vendor's weights.
  • Where open beat closed for me: I could make the model shut up. With a hosted API I would be prompting against a system tuned to always say something helpful. Here I own the loop: no citation, no verdict; too young, not an ancestor.

Numbers

Review comments ingested 2,690 (2019–2026), 168 files
Embedded (β‰₯25 chars) 2,442 with EmbeddingGemma, 768 dims
Commits indexed 1,482
Ingest wall time (CPU) 448 s
Charter 1 Gemma call, ~120 s
Review PR #3769 4 hunks, 106 s, 3 notes / 1 silent
Review PR #3767 4 hunks, 106 s, 1 note / 3 silent
Review PR #3777 2 hunks, 55 s, 2 silent
Hardware Ryzen 7 5700G, 31 GB RAM, no discrete GPU

Prize Categories

  • Best Use of Gemma β€” Gemma 3 4B generates every verdict and the charter; EmbeddingGemma does all retrieval. Both run locally through Ollama; nothing leaves the machine.
  • Best Use of Mastra β€” Agent + Memory on LibSQLStore, LibSQLVector for the ancestor index, and a three-step createWorkflow for the charter.
  • Best Use of Temporal β€” the hauntRepo workflow with heartbeating reviewPr activities; crash-and-resume demonstrated above.

Team

Solo: @vigohan95.

Disclosure: I used Claude Code as a pair programmer while building this. The design, the mistakes (see rule 1), every run and every number above are mine and real.

Top comments (0)