ChatGPT made this picture of a man at a mailbox. Then, when researchers at the Brennan Center for Justice asked six chatbots whether it was AI-made, ChatGPT studied the shadows, the hands and the street sign and called it very likely genuine. Grok said there was a real mailbox on that street in San Mateo, California. Claude guessed stock photo, and Meta AI called it real. Only Gemini read the hidden OpenAI label the file carried, the kind of label California's AI Transparency Act has required since August 2. A mid-September rerun, after the law took effect, found nothing had improved.
We also go through their bigger August study, where ChatGPT and Grok didn't flag a single image their own tools had made, plus why a screenshot can wipe that hidden label and why a missing label proves nothing. The takeaway: a chatbot saying "looks real" is a guess, not a fact-check. Check who else is reporting the image: fact-checkers like PolitiFact or AP Fact Check, and for anything about voting, your county election office.
ChatGPT called its own fake real
You'd think ChatGPT knows its own fakes. It called this very likely real. Say you're scrolling at night. A photo stops you: a man at a mailbox. Something feels off, so you ask a chatbot: is this real?
Six chatbots, one photo
Researchers at the Brennan Center for Justice did exactly that, with six chatbots and this exact picture. ChatGPT studied the shadows, the hands and the street sign. Its verdict: very likely genuine. Grok went further. It said there's a real mailbox on that street in San Mateo, California, and the houses and palm trees match. Claude saw no sign of fakery and guessed stock photo. Meta AI called it real, and Perplexity said it didn't look obviously AI.
The hidden label only Gemini read
And here's the part that stings: the file carried a hidden label from OpenAI, saying where it came from. Only Gemini did the boring, correct thing: it read that label and named OpenAI as the maker. None of the others even mentioned it. Not even ChatGPT, the one that made the picture.
The new law and the September rerun
That label isn't a nice extra. California's AI Transparency Act took effect August second, and it requires AI images to carry one. That mailbox test was the mid-September rerun, after the law took effect. Nothing had improved, and Gemini was still the only one checking.
The bigger study: 14 AI images
And this wasn't one bad day. In their earlier, bigger study of 14 AI-made images, ChatGPT, Grok, Claude and Perplexity each caught four or fewer. Gemini and Meta AI caught most. But ChatGPT and Grok didn't flag a single image their own tools had made. To be fair, after that study OpenAI released a separate checking tool, for its own images only. Retested, ChatGPT caught some of its own images. Not all. And these were a small set of test images from one research team, not every photo out there.
Why labels won't save you either
But don't crown Gemini either. It reads labels, and pictures with no readable label, it couldn't identify. And that label is fragile. OpenAI says a screenshot or a file conversion can wipe that metadata. So the photo you saw, screenshotted and reshared, may have no label left. And a missing label proves nothing, because most real photos don't have one either.
Pixels lie: how to actually check a photo
Label or no label, most of these chatbots did what you'd do: squint at pixels. And pixels lie. As the Brennan Center's Abdiaziz Ahmed put it, "a chatbot's answer is not verification." So don't zoom in. Zoom out: if that really happened, would real news outlets be covering it? Check fact-checkers like PolitiFact or AP Fact Check, and for anything about voting, your county election office. Before you hit share, remember one thing. A chatbot saying "looks real" is a guess, not a fact-check.
Sources
- Chatbots Still Bad at Spotting AI-Generated Images Despite New Law — Brennan Center for Justice
- Does AI Fight or Fuel Election Disinformation? — Brennan Center for Justice
- LLMs Are Still Making Election Deepfakes Just Days Before the US Midterms — Tech Policy Press
- Verify OpenAI-generated content — OpenAI
- Provenance signals in OpenAI-generated content — OpenAI Help Center
- California Revamps AI Transparency Requirements — Morrison Foerster
Read the full transcript, with every source, on the page for this video. More explainers like this one are on the Kush_EXE YouTube channel.
This article is built from the video's script and published sources. The video uses AI illustrations.
Top comments (0)