DEV Community

Naweid Hassan
Naweid Hassan

Posted on

I Built a Chrome Extension That Fact-Checks AI Outputs: Here's How It Works

#productivity #webdev #programming #ai

About 27% of AI outputs contain fabricated claims. I kept running into this problem — ChatGPT confidently citing studies that don't exist, Gemini inventing statistics, Claude hallucinating facts. So I built a tool to catch it.

What is Aretify?

Aretify is a Chrome extension that adds a "Verify" button to AI responses on ChatGPT, Claude, and Gemini. One click extracts every factual claim and checks it against real sources.

The Tech Stack

  • Frontend: Next.js on Vercel
  • Backend: FastAPI on Railway
  • Database: Supabase (PostgreSQL)
  • Cache: Upstash Redis
  • Extension: Chrome Manifest V3, side panel architecture

How Verification Works

When you click "Verify," here's what happens:

  1. Claim Extraction — The AI response is sent to Groq for fast claim extraction. Each individual factual claim is isolated.

  2. Evidence Search — Each claim is searched against 15+ evidence sources simultaneously: Semantic Scholar, PubMed, OpenAlex, Crossref, Europe PMC, Wikipedia, Wikidata, Google Fact Check, GDELT, NewsAPI, Brave Search, SerpAPI, and more.

  3. Scoring — Claims are scored based on how well they match evidence. Each gets a status: Verified, Partial Match, or Unverified. The overall response gets an AretifyScore from 0-100.

  4. Ethics Analysis — This is the part I'm most proud of. My father is a philosophy professor, and he developed a 35-page framework covering 8 philosophical traditions (utilitarian, deontological, virtue ethics, care ethics, and more). Every verification runs through this ethics engine to evaluate the ethical implications of AI outputs — not just accuracy.

What I Learned Building This

API redundancy matters. With 15+ evidence sources, some are always down. I implemented circuit breakers for unreliable APIs (Tavily, GDELT) so the system degrades gracefully instead of failing.

Chrome Extension development is painful. Manifest V3 restrictions, CSP issues across different AI platforms, and the fact that .ico files cause context invalidation in MV3 (Chrome requires .png) — these were hours of debugging.

Canonical URLs matter from day one. I launched with inconsistent www vs non-www URLs and Google was confused about which pages to index. Fix this before you launch, not after.

Try It

The extension is free — 10 verifications per day, no credit card required.

I'd love feedback from the dev community — especially on the verification pipeline and extension UX. What would you want from a tool like this?

Top comments (0)