DEV Community

Cover image for From Misinformation Crisis to Intelligent Solutions: Building a Multi-Agent Fact-Checking System
Nilay Jain
Nilay Jain

Posted on

From Misinformation Crisis to Intelligent Solutions: Building a Multi-Agent Fact-Checking System

The spread of misinformation has become one of the defining challenges of our digital age. Every day, false claims spread across social media platforms faster than truth can catch up. I remember reading that fake news spreads 6 times faster than accurate information on social networks, and that statistic stuck with me. It wasn't just a number—it represented real harm to public discourse and decision-making.

When I joined the 5-Day AI Agents Intensive Course by Google and Kaggle, I had a question: could we build something that fights back against this tide of misinformation at scale?

The Problem Nobody Asked For (But Everyone Needs)

The fact-checking industry faces a brutal reality: manual verification doesn't scale. Professional fact-checkers can verify maybe a handful of claims per day, while millions of articles are published daily. It's like trying to empty an ocean with a teaspoon.

I wanted to build something different—not just another chatbot, but an intelligent system that could verify claims automatically by cross-referencing multiple sources, understanding context, and providing transparent evidence for its conclusions.

What I Built

The Fake News Detection Agent is a multi-agent system that automates the entire fact-checking pipeline. Think of it as having five specialized experts working together: one gathers information, another identifies claims, a third searches for evidence, the fourth evaluates that evidence, and the fifth writes comprehensive reports.

Here's what makes it interesting:

Architecture:

Five specialized agents work in sequence, each handling a specific task. The Ingestion Agent processes input (URLs or raw text), the Claim Extraction Agent identifies the main verifiable claim, the Verification Agent searches for evidence across both a FAISS knowledge base and live Google Search results, the Aggregator Agent weighs the evidence, and the Report Agent generates detailed, human-readable reports.

Dual-Source Verification:

The system doesn't rely on a single source of truth. It combines a pre-indexed FAISS vector database (for fast semantic search) with real-time Google Search (for current events). This hybrid approach means you get both speed and freshness—like having a well-researched encyclopedia and today's newspaper at the same time.

Memory System:

Using SQLite, the agent caches previous verifications. If someone asks about a claim we've already checked, the response is instant—over 700x faster than running the full pipeline. This isn't just about speed; it's about being practical in a production environment.

Image Verification:

One feature I'm particularly proud of is the image processing capability. The agent can extract text from screenshots or memes using Gemini Vision, identify claims within that text, and verify them using the same pipeline. Misinformation often spreads through images, not just text.

Claim Clustering:

To avoid redundant verifications, I implemented a clustering agent that groups semantically similar claims before verification. If someone submits "Messi scored 2 goals" and "Lionel Messi netted twice," the system recognizes these as the same claim and processes them together.

You can explore the project here:

The Tech Stack

Building this required orchestrating several technologies:

  • Google ADK for multi-agent orchestration—this was the backbone that made sequential agent coordination possible
  • Gemini 2.5 Flash as the LLM for claim extraction, evidence evaluation, and report generation
  • FAISS for semantic search over the knowledge base (using HuggingFace's multilingual-e5-small embeddings)
  • Google Search integration through Gemini for real-time verification
  • SQLite with WAL mode for persistent memory and session management
  • Gradio for the web interface—simple, clean, and async-friendly

The architecture follows a clear data flow: Input → Claim Extraction → Evidence Retrieval (parallel FAISS + Google) → Semantic Ranking → Batch Evaluation → Verdict Aggregation → Report Generation.

What the Course Taught Me

The 5-Day AI Agents Intensive Course changed how I think about building AI systems. Before this course, I viewed AI as monolithic models that do everything. The course introduced me to a more modular paradigm: specialized agents, each excellent at one task, working together.

A few concepts really clicked for me:
Sequential vs Parallel Orchestration: I learned when to chain agents sequentially (like my fact-checking pipeline) versus when to run them in parallel (like retrieving from FAISS and Google Search simultaneously). The course's examples on tool calling patterns were invaluable here.

Memory and State Management: One of the most practical lessons was understanding how agents need memory to be effective. My implementation of session tracking and verdict caching directly came from the course's discussion on persistent agent state.

Tool Design Principles: The course emphasized making tools simple, focused, and composable. I applied this by creating small, testable tools (like faiss_search and google_search_tool) that agents could call independently rather than building monolithic functions.

Error Handling in Agent Workflows: Real-world agents fail. Networks timeout, APIs rate-limit you, search results are empty. The course's emphasis on graceful degradation helped me implement fallback mechanisms—like using cached results when fresh data isn't available.

The Capstone Project Journey

Building this wasn't linear. Early versions had a fatal flaw: they tried to verify every sub-claim individually, leading to API exhaustion and long wait times. A claim like "Dharmendra died recently at age 88 in Mumbai due to heart failure" would generate four separate verifications (person, age, location, cause).

The solution? Claim clustering. Instead of verifying fragments, the system now consolidates related claims into comprehensive statements before verification. This reduced API calls by 60-70% in testing.

Another challenge was balancing accuracy with speed. Initially, I fetched only 3 pieces of evidence per claim. This was fast but unreliable. After experimenting, I settled on retrieving 15 items (10 from Google, 5 from FAISS) and using semantic ranking to select the top 10 most relevant pieces. This hybrid approach improved verdict accuracy significantly without making the system too slow.

The binary classification system (SUPPORTS vs REFUTES) was also a hard-earned lesson. Early versions had a three-way classification (SUPPORTS, REFUTES, NOT_ENOUGH_INFO), but agents would overuse NOT_ENOUGH_INFO as a hedge. Forcing binary decisions made the system more decisive and aligned better with how fact-checking actually works.

Real-World Impact

What excites me most is the potential for scaling fact-checking. Imagine this system integrated into:

  • Social media platforms: Automatically flagging suspicious claims with evidence before they go viral
  • News aggregators: Providing credibility scores alongside headlines
  • Educational platforms: Teaching students to critically evaluate sources by showing the verification process
  • Browser extensions: Real-time fact-checking as you browse

The open-source nature of this project means anyone can adapt it. Want to verify claims in a specific domain like medical research? Swap in a specialized FAISS index. Need multilingual support? The embeddings already handle multiple languages.

Key Features Implemented

  • Sequential Multi-Agent Pipeline: Five specialized agents working in harmony
  • Hybrid Evidence Retrieval: FAISS (5 results) + Google Search (10 results) with semantic ranking
  • Smart Caching: SQLite-based memory for instant retrieval of previous verifications
  • Image OCR: Gemini Vision for extracting and verifying claims from images
  • Claim Clustering: Reduces redundant verifications by 60-70%
  • Binary Verdict System: Clear TRUE/FALSE classifications with confidence scores
  • Detailed Reporting: Transparent explanations with source attribution
  • Web Interface: Gradio-based UI with real-time agent thinking visualization

What I Learned Beyond Code

Technical skills aside, this course taught me how to think about AI differently. The shift from "what can a model do?" to "what can a system of agents do together?" is fundamental. It's the difference between having a talented individual and having a well-coordinated team.

I also learned the importance of prompt engineering for agents. The quality of my claim extraction improved dramatically when I refined the prompts to be more specific about what constitutes a "verifiable claim" versus an opinion.

Perhaps most importantly, I learned to respect the complexity of real-world deployment. Building something that works in a Jupyter notebook is one thing; building something that handles edge cases, network failures, and diverse user inputs is entirely different.

Looking Forward

This project is just a starting point. Future enhancements I'm considering to include:

  • Real-time monitoring: Track trending topics and proactively verify popular claims
  • Multi-language expansion: Extend beyond English to combat global misinformation
  • User feedback loops: Allow users to flag incorrect verdicts for continuous improvement
  • Browser extension: Bring verification directly to where people consume news
  • API endpoints: Enable other applications to integrate fact-checking capabilities

The 5-Day AI Agents Intensive Course gave me the foundation to build this, and I'm grateful for the structured approach Google and Kaggle provided. From understanding agent architectures on Day 1 to deploying a working system by the capstone, the progression was well-designed.

I would also like to thank all the coordinators for organizing this course and the livestream Q&A sessions. I truly enjoyed the discussions during the livestreams and found it fascinating to learn from the Google experts.

Top comments (0)