DEV Community

Cover image for # Reality-Check-AI An AI-powered evidence ledger for everyday claims **Vibe-Code Something Strange?**
Aguiovani Oliveira
Aguiovani Oliveira

Posted on

# Reality-Check-AI An AI-powered evidence ledger for everyday claims **Vibe-Code Something Strange?**

#sanitychallenge #ai #webdev #programming

Sanity Challenge Path Two Submission

DEMO

1. Project Summary

Reality Check AI is an AI-powered evidence verification application that helps people investigate claims they encounter online, in conversations, articles, documentation, and social media.

Instead of providing a simple AI-generated answer, Reality Check AI creates a structured Truth Card containing:

  • The original claim
  • The interpretation of the claim
  • Supporting evidence
  • Contradicting evidence
  • Source information
  • Evidence excerpts
  • AI analysis
  • Confidence level
  • Verification status
  • Human review status
  • Investigation date

The central idea is simple:

Don't just ask AI what is true. Ask AI to show you why.

Sanity is used as the structured content layer that stores claims, sources, evidence, analyses, and verification records.

The AI agent operates on this structured content and can create, investigate, compare, and update Truth Cards.

2. The Problem

AI assistants are extremely good at producing answers.

However, an answer can look convincing even when:

  • The original claim is ambiguous.
  • Evidence is weak.
  • Sources disagree.
  • The source is outdated.
  • The AI misunderstood the claim.
  • A conclusion goes beyond what the evidence actually says.

For example:

"Using dark mode saves a significant amount of battery."

A normal chatbot might answer:

"Yes, dark mode can save battery, especially on OLED displays."

But Reality Check AI asks a different question:

What evidence supports this specific claim?

It separates the claim from the evidence and preserves the relationship between them.

3. The Big Idea

Reality Check AI creates an evidence graph.

                    ┌───────────────────┐
                    │       CLAIM       │
                    │                   │
                    │ "Dark mode saves  │
                    │ battery."         │
                    └─────────┬─────────┘
                              │
                 ┌────────────┴────────────┐
                 │                         │
                 ▼                         ▼
        ┌─────────────────┐       ┌─────────────────┐
        │ SUPPORTING      │       │ CONTRADICTING   │
        │ EVIDENCE        │       │ EVIDENCE        │
        └────────┬────────┘       └────────┬────────┘
                 │                         │
                 ▼                         ▼
        ┌─────────────────┐       ┌─────────────────┐
        │ SOURCE          │       │ SOURCE          │
        └─────────────────┘       └─────────────────┘
                 │
                 ▼
        ┌─────────────────┐
        │ AI ANALYSIS     │
        └────────┬────────┘
                 │
                 ▼
        ┌─────────────────┐
        │ TRUTH CARD      │
        │                 │
        │ Supported       │
        │ Confidence: 82% │
        └─────────────────┘
Enter fullscreen mode Exit fullscreen mode

The application therefore does not treat AI output as the final source of truth.

The evidence remains visible.

4. Example Investigation

A user enters:

"Learning to code is becoming useless because AI can write code."

Reality Check AI creates an investigation.

Claim

Learning to code is becoming useless because AI can write code.

Claim interpretation

The agent identifies that the statement contains multiple ideas:

  1. AI can generate software code.
  2. AI reduces the amount of manual coding required.
  3. Programming knowledge is therefore becoming unnecessary.

The third statement does not automatically follow from the first two.

Evidence

The agent retrieves relevant sources and stores individual evidence records.

Evidence #1

Type:
Supporting

Source:
Official / authoritative technical source

Excerpt:
[Relevant source passage]

Location:
Section / page / URL

Analysis:
This supports the claim that AI can automate
some coding activities, but does not establish
that programming knowledge is unnecessary.
Enter fullscreen mode Exit fullscreen mode

Another record might contain:

Evidence #2

Type:
Contradicting / limiting

Source:
Technical documentation or research

Excerpt:
[Relevant source passage]

Analysis:
The source describes human involvement in
software development and therefore limits the
stronger interpretation of the claim.
Enter fullscreen mode Exit fullscreen mode

Final Truth Card 

┌──────────────────────────────────────┐
│          REALITY CHECK               │
├──────────────────────────────────────┤
│                                      │
│ Learning to code is becoming         │
│ useless because AI can write code.   │
│                                      │
│ STATUS                               │
│ ⚠ PARTIALLY SUPPORTED                │
│                                      │
│ The evidence supports the claim that │
│ AI can automate some coding tasks.   │
│                                      │
│ It does not establish that learning  │
│ programming has become unnecessary.  │
│                                      │
│ Evidence: 6 sources                  │
│ Reviewed: AI + Human                 │
│ Confidence: Medium                   │
└──────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

5. Why AI Is Important

AI is not simply used as a chatbot interface.

The AI acts as an investigation agent.

It performs several tasks:

Claim Parser

Converts an informal statement into a structured claim.

Claim Decomposer

Breaks complicated claims into smaller propositions.

Evidence Finder

Identifies relevant sources.

Evidence Extractor

Extracts the specific passages relevant to the claim.

Evidence Analyzer

Determines what each source actually supports.

Contradiction Detector

Looks for evidence that conflicts with the claim.

Verdict Generator

Produces a conclusion based on the available evidence.

Citation Agent

Maintains the relationship between the conclusion and the evidence.

6. The Sanity Content Model

Sanity is the core structured-content layer.

The project uses several document types.

6.1 Claim 

claim
├── title
├── statement
├── normalizedClaim
├── category
├── topic
├── createdAt
├── status
├── verdict
├── confidence
├── summary
└── slug
Enter fullscreen mode Exit fullscreen mode

Example:

{
  "_type": "claim",
  "statement": "Learning to code is becoming useless because AI can write code.",
  "category": "Technology",
  "status": "investigated",
  "verdict": "partially-supported",
  "confidence": 0.82
}
Enter fullscreen mode Exit fullscreen mode

6.2 Source 

source
├── title
├── publisher
├── sourceType
├── url
├── publicationDate
├── jurisdiction
├── credibilityNotes
└── accessedAt
Enter fullscreen mode Exit fullscreen mode

A source can be:

  • Government documentation
  • University research
  • Technical documentation
  • Scientific paper
  • Company documentation
  • Standards organization
  • Official statistics
  • Other authoritative material

6.3 Evidence

Evidence connects a claim to a source.

evidence
├── title
├── claim
├── source
├── excerpt
├── location
├── evidenceType
├── relevance
├── confidence
├── interpretation
├── extractedBy
└── reviewedAt
Enter fullscreen mode Exit fullscreen mode

This relationship is the heart of the application.

Claim
  ↓
Evidence
  ↓
Source
Enter fullscreen mode Exit fullscreen mode

6.4 Investigation

An investigation records the AI agent's work.

investigation
├── claim
├── question
├── startedAt
├── completedAt
├── agentVersion
├── sourcesExamined
├── evidenceFound
├── reasoningSummary
└── finalVerdict
Enter fullscreen mode Exit fullscreen mode

This allows the system to preserve how an investigation was performed.

6.5 Review

Human review can be represented separately.

review
├── claim
├── reviewer
├── decision
├── comments
├── reviewedEvidence
└── reviewDate
Enter fullscreen mode Exit fullscreen mode

Possible decisions:

approved
needs-review
rejected
outdated
disputed
Enter fullscreen mode Exit fullscreen mode

7. Verification States

The application does not force everything into TRUE/FALSE.

Possible statuses are:

SUPPORTED
PARTIALLY_SUPPORTED
UNCLEAR
CONTRADICTED
OUTDATED
DISPUTED
INSUFFICIENT_EVIDENCE
Enter fullscreen mode Exit fullscreen mode

This is important because real-world claims are often more complicated than binary answers.

8. Confidence

Each investigation receives a confidence level.

HIGH
MEDIUM
LOW
Enter fullscreen mode Exit fullscreen mode

The confidence is based on factors such as:

  • Number of relevant sources
  • Source authority
  • Agreement between sources
  • Evidence specificity
  • Source freshness
  • Claim ambiguity
  • Contradictory evidence

The application should clearly distinguish:

Evidence confidence

from:

Truth itself

The system is reporting the strength of the available evidence, not claiming perfect knowledge.

9. User Experience

The homepage contains a simple input:

┌────────────────────────────────────────────┐
│ What claim do you want to investigate?    │
│                                            │
│ "Does cold water make you burn more       │
│ calories?"                                 │
│                                            │
│             [ Investigate ]                │
└────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The AI then displays:

Analyzing claim...

✓ Understanding claim
✓ Breaking claim into propositions
✓ Finding relevant sources
✓ Extracting evidence
✓ Comparing evidence
✓ Generating analysis
✓ Creating Truth Card
Enter fullscreen mode Exit fullscreen mode

Then:

                 REALITY CHECK

        PARTIALLY SUPPORTED

        Confidence: MEDIUM

        ───────────────────────────

        Evidence supporting claim
        3 sources

        Evidence limiting claim
        2 sources

        Sources examined
        7

        Human review
        Pending
Enter fullscreen mode Exit fullscreen mode

10. Claim Detail Page

Each claim gets its own URL:

/claims/learning-to-code-ai
Enter fullscreen mode Exit fullscreen mode

The page contains:

Claim

The original user statement.

Interpretation

What the AI believes the claim means.

Verdict

The current evidence status.

Evidence Timeline 

Source discovered
       ↓
Evidence extracted
       ↓
AI analysis
       ↓
Human review
       ↓
Verification
Enter fullscreen mode Exit fullscreen mode

Supporting Evidence

Exact excerpts.

Contradicting Evidence

Exact excerpts.

Sources

Links to original sources.

Reviewer Notes

Human analysis.

Investigation History

Previous versions of the investigation.

11. AI Agent Architecture 

                 USER
                   │
                   ▼
            ┌─────────────┐
            │ Next.js UI  │
            └──────┬──────┘
                   │
                   ▼
            ┌─────────────┐
            │ AI AGENT    │
            └──────┬──────┘
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
   Claim Tool  Search Tool  Sanity Tool
        │          │          │
        └──────────┼──────────┘
                   ▼
             ┌───────────┐
             │ Sanity    │
             │ Content   │
             │ Lake      │
             └───────────┘
                   │
                   ▼
              GROQ Query
                   │
                   ▼
              Truth Card
Enter fullscreen mode Exit fullscreen mode

12. AI Tools

The agent can have tools such as:

createClaim()
searchSources()
createEvidence()
analyzeEvidence()
findContradictions()
updateClaim()
createInvestigation()
requestHumanReview()
Enter fullscreen mode Exit fullscreen mode

For example:

User
 ↓
"Investigate whether dark mode saves battery."

AI
 ↓
createClaim()

AI
 ↓
searchSources()

AI
 ↓
createEvidence()

AI
 ↓
analyzeEvidence()

AI
 ↓
findContradictions()

AI
 ↓
createInvestigation()

AI
 ↓
generateTruthCard()
Enter fullscreen mode Exit fullscreen mode

13. GROQ

Sanity's GROQ queries allow the frontend and AI agent to retrieve structured relationships.

For example:

*[
  _type == "claim" &&
  slug.current == $slug
][0]{
  title,
  statement,
  verdict,
  confidence,
  summary,
  "evidence": *[
    _type == "evidence" &&
    references(^._id)
  ]{
    title,
    excerpt,
    location,
    evidenceType,
    confidence,
    interpretation,
    "source": source->{
      title,
      publisher,
      url
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

This produces the complete evidence chain for a claim.

14. Technology Stack

Frontend

  • Next.js
  • React
  • TypeScript
  • Tailwind CSS

Content

  • Sanity
  • Sanity Studio
  • Content Lake
  • GROQ

AI

  • AI agent
  • LLM
  • Structured tool calls
  • Source analysis
  • Claim decomposition

Deployment

  • Vercel
  • Sanity hosting

Development

  • VS Code
  • Git
  • GitHub
  • PowerShell

15. Suggested Repository Structure 

reality-check-ai/
│
├── app/
│   ├── page.tsx
│   ├── investigate/
│   │   └── page.tsx
│   ├── claims/
│   │   └── [slug]/
│   │       └── page.tsx
│   └── api/
│       └── investigate/
│           └── route.ts
│
├── components/
│   ├── ClaimCard.tsx
│   ├── EvidenceCard.tsx
│   ├── SourceCard.tsx
│   ├── VerdictBadge.tsx
│   ├── InvestigationProgress.tsx
│   └── TruthCard.tsx
│
├── sanity/
│   ├── schemas/
│   │   ├── claim.ts
│   │   ├── source.ts
│   │   ├── evidence.ts
│   │   ├── investigation.ts
│   │   └── review.ts
│   └── lib/
│       └── client.ts
│
├── lib/
│   ├── ai/
│   ├── groq/
│   └── verification/
│
├── public/
│
└── README.md
Enter fullscreen mode Exit fullscreen mode

16. Demo Scenario

For the challenge demo, use a claim that is immediately understandable.

Demo claim

"AI-generated code means programmers no longer need to understand programming."

The demo starts with:

NEW INVESTIGATION

Claim:
AI-generated code means programmers no longer
need to understand programming.

[ INVESTIGATE ]
Enter fullscreen mode Exit fullscreen mode

Then show the AI working.

✓ Claim interpreted

✓ Claim decomposed

✓ Sources identified

✓ Evidence extracted

✓ Evidence compared

✓ Contradictions checked

✓ Truth Card generated
Enter fullscreen mode Exit fullscreen mode

Then reveal:

┌───────────────────────────────────────┐
│             TRUTH CARD                │
│                                       │
│ PARTIALLY SUPPORTED                   │
│                                       │
│ AI can automate portions of software  │
│ development, but the available       │
│ evidence does not establish that      │
│ programming knowledge is unnecessary. │
│                                       │
│ Supporting evidence: 4                │
│ Limiting evidence: 5                  │
│ Sources: 8                            │
│ Confidence: Medium                    │
│                                       │
│ [VIEW EVIDENCE]                       │
└───────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

Click VIEW EVIDENCE.

The user can then see exactly which source passages produced the conclusion.

That is the key visual moment of the demo.

17. What Makes the Project Different

The project is not:

"Ask ChatGPT a question."

It is:

"Build a persistent, structured evidence trail around an AI investigation."

The AI's answer becomes only one part of the system.

The important objects are:

Claim
Source
Evidence
Investigation
Review
Verdict
Enter fullscreen mode Exit fullscreen mode

These objects are persistent Sanity documents.

That means the system can remember previous investigations and improve them over time.

18. Strange Feature: "Evidence Court"

For the fun part of the challenge, Reality Check AI can include an Evidence Court.

When evidence conflicts, the application visually presents a miniature courtroom.

             ⚖ EVIDENCE COURT

        ┌─────────────┐
        │    CLAIM    │
        └──────┬──────┘
               │
       ┌───────┴────────┐
       ▼                ▼
  SUPPORTS IT       CHALLENGES IT
       │                │
       ▼                ▼
   Evidence A       Evidence B
   Evidence C       Evidence D
       │                │
       └───────┬────────┘
               ▼
        AI EVIDENCE REVIEW
               │
               ▼
          FINAL STATUS
Enter fullscreen mode Exit fullscreen mode

The "court" does not pretend that AI is an actual judge.

It is simply a memorable interface for showing competing evidence.

19. Dashboard

The application can also provide:

REALITY CHECK DASHBOARD

Investigations       127
Supported             42
Partially Supported   51
Unclear               19
Contradicted           9
Needs Review           6
Enter fullscreen mode Exit fullscreen mode

Additional statistics:

  • Most investigated topics
  • Most frequently disputed claims
  • Sources used most often
  • Claims awaiting human review
  • Recently updated investigations
  • Evidence confidence distribution

20. Search

Users can search:

"AI"
"battery"
"programming"
"exercise"
"productivity"
"sleep"
Enter fullscreen mode Exit fullscreen mode

The search results can include:

Claim
Verdict
Confidence
Evidence count
Last verified
Enter fullscreen mode Exit fullscreen mode

21. Filters

The dashboard supports:

Category
├── Technology
├── Science
├── Programming
├── Productivity
├── Business
└── Everyday Life

Status
├── Supported
├── Partially Supported
├── Unclear
├── Contradicted
└── Needs Review
Enter fullscreen mode Exit fullscreen mode

22. Human-in-the-Loop

AI should not silently become the authority.

The application therefore supports human review.

An investigator can open a claim and see:

AI VERDICT

PARTIALLY SUPPORTED

[ Approve ]
[ Request Review ]
[ Reject ]
Enter fullscreen mode Exit fullscreen mode

A reviewer can write:

"The evidence supports the first part of the
claim but does not establish the stronger
conclusion."
Enter fullscreen mode Exit fullscreen mode

That review becomes structured Sanity content.

23. Why Sanity Fits

Sanity is particularly useful because the application contains relationships between many types of structured information.

Instead of one giant document:

claim + sources + evidence + review
Enter fullscreen mode Exit fullscreen mode

the system keeps them as separate reusable objects:

Claim
Source
Evidence
Investigation
Review
Enter fullscreen mode Exit fullscreen mode

This makes it possible to:

  • Reuse sources across multiple claims
  • Reuse evidence across investigations
  • Update claims independently
  • Track verification history
  • Query relationships with GROQ
  • Build different frontend views
  • Create AI workflows around structured content

24. MVP

The first version should remain small.

MVP features

  1. Create a claim
  2. AI interprets the claim
  3. AI creates an investigation
  4. Sources are attached
  5. Evidence records are created
  6. AI analyzes evidence
  7. Truth Card is generated
  8. Human reviewer can approve it
  9. Claim page displays the complete evidence chain
  10. Search and filtering work

That is enough for a compelling challenge submission.

25. Phase 2

Later versions can add:

  • Claim history
  • Multiple reviewers
  • Source freshness monitoring
  • Automatic re-verification
  • Contradiction graphs
  • User accounts
  • Public investigations
  • Community review
  • Evidence voting
  • Browser extension
  • API access
  • Export to Markdown/PDF
  • AI-generated investigation reports

26. Phase 3 — The Agent

Eventually, the system can become proactive.

For example:

SOURCE CHANGED

A source used by 12 claims has changed.

Would you like Reality Check AI to
re-investigate those claims?

[ INVESTIGATE 12 CLAIMS ]
Enter fullscreen mode Exit fullscreen mode

The agent can automatically:

Detect source change
        ↓
Find affected claims
        ↓
Re-run investigations
        ↓
Compare previous evidence
        ↓
Generate updated verdicts
        ↓
Request human review
Enter fullscreen mode Exit fullscreen mode

This turns the application from a static database into an AI-powered verification system.

27. The Core Data Philosophy

The most important design principle is:

Every conclusion should be traceable.

Instead of:

AI says: TRUE
Enter fullscreen mode Exit fullscreen mode

the system should show:

AI says: SUPPORTED

Because:

Evidence #1
    ↓
Source #1
    ↓
Exact excerpt
    ↓
AI interpretation

Evidence #2
    ↓
Source #2
    ↓
Exact excerpt
    ↓
AI interpretation
Enter fullscreen mode Exit fullscreen mode

This creates an auditable chain.

28. Tagline

Primary tagline

Don't just trust the answer. Check the evidence.

Other possibilities:

AI that shows its receipts.

Every claim deserves evidence.

Turn AI answers into evidence trails.

Ask a question. Get the receipts.

The last one would work particularly well for the demo:

Reality Check AI

Ask a question. Get the receipts.

29. Challenge Submission Description

Reality Check AI is an AI-powered evidence ledger for investigating everyday claims.

Instead of returning a simple AI-generated answer, the application decomposes a claim, identifies relevant sources, extracts specific evidence, analyzes supporting and contradicting information, and produces a structured Truth Card.

Sanity stores the claims, sources, evidence, investigations, and human reviews as interconnected documents. GROQ retrieves these relationships for the Next.js application.

The core data relationship is:

Claim → Evidence → Source → Analysis → Review → Verdict

The project demonstrates how structured content can become more than a CMS. Sanity acts as the persistent evidence layer for an AI agent, allowing investigations to remain inspectable, searchable, and reusable after the original AI conversation ends.

The project's central principle is:

Don't just trust the answer. Check the evidence.

30. Final Architecture 

                         USER
                           │
                           ▼
                  ┌─────────────────┐
                  │    NEXT.JS      │
                  │   APPLICATION   │
                  └────────┬────────┘
                           │
                           ▼
                  ┌─────────────────┐
                  │   AI AGENT      │
                  │                 │
                  │ Claim Parser    │
                  │ Researcher      │
                  │ Evidence Agent  │
                  │ Analyst         │
                  │ Reviewer        │
                  └────────┬────────┘
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
          Claim Tool   Search Tool  Sanity Tool
              │            │            │
              └────────────┼────────────┘
                           ▼
                  ┌─────────────────┐
                  │ SANITY CONTENT  │
                  │      LAKE       │
                  ├─────────────────┤
                  │ Claims          │
                  │ Sources         │
                  │ Evidence        │
                  │ Investigations  │
                  │ Reviews         │
                  └────────┬────────┘
                           │
                           ▼
                         GROQ
                           │
                           ▼
                  ┌─────────────────┐
                  │   TRUTH CARD    │
                  │                 │
                  │ Verdict         │
                  │ Evidence        │
                  │ Sources         │
                  │ Confidence      │
                  │ Review          │
                  └─────────────────┘
Enter fullscreen mode Exit fullscreen mode

31. The One-Sentence Pitch

Reality Check AI is a Sanity-powered AI agent that investigates claims, preserves the evidence behind its conclusions, and lets humans inspect exactly how an AI arrived at an answer.

Top comments (1)

Collapse
 
snocterry profile image
Aguiovani Oliveira •

Reality Check AI is needed in various range of our life