This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
A source changes. The claim based on it stays.
That is the problem behind Dobara, a local AI tool for checking whether an earlier and a current source excerpt support the same claim.
Consider a fictional scholarship notice. Its earlier version says:
Applications are open to all undergraduate students.
You write: “All undergraduate students can apply.”
Later, the notice changes:
Applications are open only to final-year undergraduate students.
Your sentence still exists in a spreadsheet, an article, or a message someone forwards. Its original evidence has become a ghost.
That is my interpretation of Ghost in the Machine: old evidence continuing to influence what we believe after the source has changed.
Dobara asks you to supply a claim and two source excerpts. It assesses the claim against each excerpt, shows the original evidence passages, and highlights wording differences. You can save a comparison, use its current source as the next baseline, and record your own review decision.
I wanted something useful for students checking opportunity notices, writers revisiting citations, and researchers reviewing source changes. The result is a focused prototype for reviewing supplied evidence. It does not authenticate pasted text, monitor websites, or establish whether a claim is universally true.
Demo
Start with Eligibility changed, run the comparison, and inspect the evidence alongside the text differences. The other fictional examples show an unchanged supporting source and a claim that the earlier excerpt never supported.
The public demo runs through a temporary Cloudflare Tunnel. Inference happens on my laptop through Ollama, so availability depends on my laptop and tunnel remaining online. I tested the public link from a separate phone and successfully ran a sample comparison.
Please use fictional information. The demo has no separate user accounts, so saved comparisons and reviews are shared among visitors. For personal material, run your own local installation using the repository instructions.
Code
GitHub repository: shoriful-mynul/dobara
The application is released under the MIT licence. The repository includes setup instructions, tests, testing limitations, and upgrade notes.
How I Built It
Dobara uses Qwen3 4B, an open-weight model running through Ollama. A Python FastAPI backend handles analysis and persistence, while the interface uses HTML, CSS, and JavaScript. Optional saved comparisons and human reviews live in a local SQLite database.
The model provides the language interpretation at the core of the application: does the supplied excerpt support the full claim, explicitly contradict it, omit the necessary information, or leave the answer uncertain?
The earlier and current excerpts are assessed independently. This matters because the earlier source might never have supported the claim in the first place.
One early problem shaped the implementation. A model-generated quotation failed validation because it did not exactly match the source. I changed the workflow so the model selects evidence IDs rather than writes quotations.
The backend now:
- Assigns IDs to the supplied evidence passages.
- Requests structured assessments and evidence IDs from Qwen3.
- Validates the response and checks the selected IDs.
- Builds quotations directly from the original passages.
- Derives the overall verdict using explicit Python rules.
This keeps quotations traceable to the input. It does not guarantee that the model interpreted them correctly.
Text differences are computed separately using Python’s difflib. A wording change and a change in support are different questions, so the interface presents both.
Human review is also separate from the AI assessment. A reviewer can choose Keep claim, Revise claim, or Find more evidence, then save a note without changing the model’s original result or running inference again. Linked rechecks create new records rather than overwrite earlier comparisons.
The release’s automated checks include 80 passing backend/static tests, plus an optional DOM interaction script. Inference is mocked in those checks; they test application behavior, not model accuracy.
I used ChatGPT/Codex and Claude for development assistance.
Why Does Open Innovation Matter?
Source excerpts can contain information someone would rather keep on their own machine. Local inference makes that possible without requiring a hosted model API for every comparison.
After setup, the application uses installed model weights through Ollama. There is no per-request model API charge, although the computer still supplies the memory, processing time, and electricity. The repository distinguishes local operation from disconnected operation, which needs testing on the target machine.
Open weights also give this project a path beyond one provider. A developer can adapt the model configuration and evaluate alternatives against the same evidence-review workflow. Better models can be tested without discarding the application’s saved-history and human-review design.
For Dobara, the practical advantage is control over where source text goes and how the assessment works. A hosted API could perform similar reasoning, but this approach lets someone own the inference environment and inspect the surrounding application code.
The public demo makes a different trade-off: requests pass through Cloudflare to my laptop. The privacy benefit belongs to running your own local copy.
The Spirit of It
The most useful moment in Dobara is small: seeing “all undergraduate students” beside “only final-year undergraduate students,” with the original wording available to inspect.
Then there is the less obvious case: both excerpts can be identical and still fail to support the claim. Nothing changed in the source. The assumption was already there.
That is the ghost I wanted to make visible.
Dobara gives a claim a second look, with the evidence still in view.
Top comments (0)