Ghost Reviewer and Hindsight: Giving AI Code Review a Long-Term Memory

Their code review instincts didn't.
By the Ghost Reviewer team: Syed Faiz, Vaibhav, Vaishnavi Pandey and Srinivas Barik
Introduction
AI coding assistants have changed how developers write, debug and maintain software. They explain unfamiliar code, generate tests, suggest improvements and flag potential bugs. But many AI-powered review workflows share one limitation: every new review is an isolated task.
Imagine a senior engineer finds a critical payment-processing issue during a review. The team fixes it, discusses the root cause and documents the lesson. A few weeks later, another developer introduces a similar mistake in a different part of the application.
Will the AI reviewer remember what happened before?
Without access to relevant engineering history, it may repeat the same investigation or miss the lesson entirely. That is the problem we set out to explore with Ghost Reviewer, an AI-powered code review assistant built around long-term engineering memory using Hindsight.
Our idea is simple. A code reviewer should understand the code submitted today, and it should also be able to retrieve engineering knowledge from previous reviews, incidents, conventions and developer feedback. The goal is to help teams keep their collective experience and make it useful in future reviews.

1. Why AI code review needs memory Consider a payment service that retries requests when network failures occur. A developer adds retry logic. At first glance the code looks correct, but the team once hit a problem where retrying a payment request without a stable idempotency key led to duplicate transaction attempts. A reviewer that only sees the current diff has no way to know that. This shows the difference between context and memory. A context window helps a model process the information available in one interaction. Long-term agent memory lets information be retained and retrieved across separate interactions. For engineering teams, useful memories include: • Previous code review comments • Production incidents and their root causes • Project-specific coding conventions • Reviewer preferences • Repeated mistakes and how they were resolved • Lessons learned from debugging The challenge is not storing everything. It is retrieving the right information at the moment a relevant code change appears.
2. Introducing Ghost Reviewer Ghost Reviewer turns engineering history into review context. A developer submits a Git diff. The application parses the changed files and line numbers, retrieves potentially relevant engineering memories and uses Google Gemini to generate actionable review comments. Developers can accept or reject each comment and give a reason. The system also includes adaptive suppression. When a team repeatedly rejects the same kind of low-value comment, Ghost Reviewer creates a project-level rule to reduce similar comments in future. Critical findings are protected by a safety guard. The core workflow is: Submit diff → Parse changes → Retrieve memories → Generate findings → Review feedback → Adapt future reviews Ghost Reviewer is not meant to replace senior developers. It is meant to make their experience more accessible and reusable. [Screenshot: Overview dashboard showing total reviews, average risk score, open findings, suppressed findings and context memories]
3. Architecture: connecting code review with long-term memory The project is split into separate services, each responsible for one part of the workflow. Frontend: React and Vite. The interface for authentication, dashboards, new reviews, review results, history, analytics, team memory, suppression rules and settings. Backend: Node.js and Express. Manages users, projects, reviews, comments, feedback, analytics and adaptive suppression rules, and coordinates requests to the AI service. AI service: Python and FastAPI. Handles unified diff parsing, Hindsight memory ingestion and recall, Gemini review generation, evidence checking and heuristic risk scoring. MongoDB Atlas. The application database. It stores users, projects, reviews, comments, feedback, rules and memory references. Hindsight. The long-term memory layer, responsible for retaining and retrieving engineering knowledge. Google Gemini. Generates review findings from the code changes and the retrieved context. The split between MongoDB and Hindsight is intentional. MongoDB stores application records. Hindsight stores and retrieves engineering memories. MongoDB keeps a Hindsight reference for each memory instead of duplicating the memory store.
4. How Hindsight fits into the review process Engineering knowledge enters through the application's memory workflow. The AI service synchronizes those records with Hindsight and can retrieve relevant items during a code review. For a new review:
5. The developer submits a unified Git diff.
6. The Node.js backend creates and manages the review request.
7. The Python service parses the diff and identifies changed files and lines.
8. The AI service retrieves potentially relevant engineering memories from Hindsight.
9. Gemini uses the available context to generate findings.
10. Evidence checking ensures historical citations refer to memories that were actually retrieved.
11. Findings go back to the backend and are stored in MongoDB.
12. The frontend shows the results and the feedback controls. The AI service also exposes dedicated memory operations: • POST /api/v1/memories/ingest • POST /api/v1/memories/search • GET /api/v1/memories/health Keeping these separate means memory operations can be handled independently of the frontend and the application database. One implementation detail matters a lot: Hindsight must actually be reachable. If it is not, the AI service can fall back to an in-memory list. That fallback is temporary and loses its contents when the service restarts. A successful review without a reachable Hindsight server must therefore never be mistaken for a successful persistent-memory operation.
13. A practical example: remembering a payment mistake To demonstrate the idea, Ghost Reviewer includes a payment-retry example built on synthetic engineering memories. Imagine the memory contains this fictional lesson: Previous payment-processing issue: retries without a stable idempotency key caused duplicate transaction attempts. Ensure retries reuse the same key and that the endpoint safely handles repeated requests. Now a developer submits a diff that adds payment retry functionality. The reviewer can retrieve the relevant memory and check whether the new implementation accounts for idempotency. A finding might read: Potential duplicate payment risk. The retry path should be checked for a stable idempotency key. A previous engineering lesson identified duplicate transaction attempts as a risk when retries are not idempotent. Verify that each retry reuses the same key and that the payment endpoint handles repeated requests safely. This is far more useful than "add error handling." It connects the current change to a relevant engineering lesson. The memory is not proof that the current code is defective, though. The reviewer still has to evaluate the actual diff. The project ships with 23 synthetic memory records: 12 historical review comments, 5 incidents, 3 conventions and 3 reviewer preferences. All of them are explicitly labelled as synthetic demo data. None is real company history.
14. Evidence-backed findings and reviewer voices A memory-aware reviewer has to be careful about how it presents history. Ghost Reviewer offers three reviewer voices: • Strict: detailed scrutiny and potential issues • Pragmatic: practical, actionable improvements • Security: security-related risks first These voices change tone and priorities, never historical facts. The application also follows evidence-integrity principles. Historical claims should cite memory references that were actually retrieved. If no relevant evidence exists, the reviewer says so instead of inventing an incident or citation. Diff parsing uses exact file and line information from unified diffs, and line numbers that are not present in the diff are dropped rather than invented. The risk score runs from 0 to 100. It is a deterministic heuristic meant to help prioritize reviews. It is not a statistically validated probability of a defect, and the interface says so wherever the score appears.
15. Learning from developer feedback Generating comments is only half of code review. Knowing which comments developers find useful matters just as much. Developers can accept or reject each finding with a reason such as false_positive, too_obvious, performance_not_critical, not_relevant, security_concern or other. The adaptive suppression mechanism looks for two qualifying rejections with the same project, category and reason before it creates a suppression rule. If a team keeps rejecting a low-value style comment for the same reason, matching future comments can be suppressed. This comes with a hard safety boundary. Critical or high-severity findings, security findings, and anything involving authentication, payments, idempotency, data loss, secrets or concurrency must never be suppressed. Suppressed comments are stored, not deleted, and developers can inspect, disable and restore rules. The aim is better relevance without letting feedback silence important engineering risks. [Screenshot: Adaptive Suppression Rules page with the deterministic safety guarantee]
16. Testing and implementation lessons Building this has shown us the gap between implementing individual services and integrating them into a reliable application. Ghost Reviewer includes backend, frontend and Python service tests. The Python tests mock Gemini and Hindsight calls, so much of the review logic can be tested without repeatedly calling external APIs. A demo walkthrough in backend/DEMO.md uses a payment-retry diff: reject a low-value style comment twice for the same reason, submit the diff again, and watch the style comment disappear while the critical payment finding stays visible. But mocked tests do not prove that external services work in production. A live Hindsight server, a valid Gemini configuration, correct service credentials and connectivity between deployed services all still need to be verified. The documentation lists the operational requirements: • Hindsight must be running for persistent memory. • The Gemini model ID must be supported by the configured API key. • INTERNAL_API_SECRET must be set in both the backend and the AI service. • Production deployments must be tested using their real service URLs. These are not minor deployment details. They decide whether the whole memory-assisted review workflow works.
17. Where we are today The application interface is running: authentication, the overview dashboard, review history, analytics, engineering memory and suppression rules are all in place. In our latest local run, the two sample payment-API reviews submitted through the interface ended with a Failed status, with zero findings and zero context memories loaded on the dashboard. That tells us the end-to-end path from diff to memory-backed finding is not yet confirmed. We are tracing it through the checklist above: Hindsight reachability, the Gemini model ID and key, the shared internal secret, and the service URLs. We would rather say this plainly than present the demo as finished. A memory system earns trust only when a retrieved memory can be shown to have shaped a finding. [Screenshot: Review History showing the two payments-api reviews and their status]
18. What comes next Our next priority is validating the complete application on deployed services. We want to confirm that a memory can be ingested, retrieved during a new review, used to produce an evidence-backed finding, and linked to developer feedback. We also want to measure whether memory-informed reviews are actually more useful than reviews without retrieved context. That evaluation should look beyond the number of comments generated. It should consider relevance, false positives, useful historical references, and whether repeated low-value comments decrease without hiding important risks. Conclusion Engineering teams build up knowledge through code reviews, production incidents, debugging and conventions. Too often that knowledge is not available when a new change needs to be reviewed. Ghost Reviewer explores a practical way to connect that experience with AI-powered review. By combining Python-based diff analysis, Gemini review generation, Hindsight long-term memory, evidence checking and human feedback, we aim to make code reviews more contextual and consistent. Hindsight provides the memory layer. Ghost Reviewer provides the engineering workflow around it. Our belief is simple: an AI reviewer should not only understand the code in front of it. It should also have a responsible way to use the engineering lessons that came before it. Project: Ghost Reviewer GitHub: https://github.com/Syed-faiz05/Ghost-reviewer Memory system: Hindsight Hindsight GitHub: https://github.com/vectorize-io/hindsight Documentation: https://hindsight.vectorize.io/