DEV Community

Bytecut
Bytecut

Posted on

How to Check a ChatGPT Statistic Before You Share It

This article was drafted with AI assistance and fact-checked against the source article.

Your next ChatGPT statistic could be invented. A researcher writing on Peec AI's blog bought a new domain, stateglobe.com, and generated 50,000 statistics pages in a single day with GPT-4.1, for about $50 in total. In the author's words: "every number was totally invented. I never checked a single one."

What happened next

  • Google buried the site almost immediately.
  • ChatGPT kept reading it. In the last 30 days, around 72,000 requests came from ChatGPT-User, the agent ChatGPT uses to open a page in real time while answering someone.
  • People clicked through. Since March, GA4 shows 10,000+ sessions from ChatGPT alone.
  • The numbers spread. The made-up figures ended up in an academic paper and in a bank's investment research.

The article's explanation: the system fetches a few pages, pulls the relevant passages, and starts to write. "Fact-checking doesn't happen anywhere in the process."

A 4-step check before you paste a number (our suggestion)

  1. Open the page the answer cites. If there is no citation, treat the number as unverified.
  2. Find who measured it. A stats page that only lists numbers, with no method or original study, is not a primary source.
  3. Search for the same figure elsewhere. A real statistic usually traces back to one original report.
  4. Leave it out if you cannot trace it. A missing number costs less than a false one.

The source article puts it simply: "Verifying sources should become an essential part of writers' and researchers' workflows."

Source

Top comments (0)