6 Tricks to Get AI to Actually Admit When It Doesn't Know Something (Instead of Making Stuff Up)

You know that person in every meeting who will confidently cite a quarterly sales figure they absolutely pulled from thin air rather than admit they didn't prepare the report? The one who says "Q3 revenue was up 23% in the Southeast region" with such conviction that everyone writes it down, even though they're completely winging it?

That's your AI chatbot.

These systems are trained to always have an answer, to complete every sentence, to sound authoritative no matter what. When you ask a question, the technology behind tools like ChatGPT predicts what words should come next based on patterns it learned from billions of text examples. Notice I said "should come next," not "are actually true." It has no concept of truth. It's just really, really good at producing text that sounds like it belongs in the conversation.

This leads to what researchers call hallucinations, which is a polite term for "confidently making stuff up." Ask ChatGPT for a specific scientific citation, and it might give you a perfectly formatted reference, complete with author names, journal title, publication date, and volume number. The citation will look professional. It will sound legitimate. And there's a decent chance the paper doesn't exist.

A lawyer in New York learned this the hard way when he used ChatGPT to research case law and submitted legal briefs citing completely fabricated cases with realistic-sounding names and dates. The judge was not amused. The cases seemed plausible because the AI had seen thousands of real legal citations and knew exactly what pattern to follow. It just didn't know (or care) whether these particular cases were real.

So how do you get your overconfident middle manager to actually say "I don't have that data" instead of improvising their way through the presentation? Here are six techniques that actually work.

Give Permission to Say 'I Don't Have That Data'

The simplest fix is to explicitly tell the AI it's allowed to admit uncertainty. Add a line to your prompt like "If you're not certain, please say so" or "Only answer if you have reliable information on this topic."

Without that permission, the system defaults to its training: always complete the response, always sound helpful, always give the person what they seem to be asking for. With permission, it can take the out you've offered.

Try asking for restaurant recommendations in a city you're visiting. "What are the best sushi restaurants in Boise?" will get you a confident list, possibly featuring establishments that closed in 2019 or never existed. Add "If you don't have current, reliable information about Boise restaurants, please tell me instead of guessing," and watch the response shift to something much more hedged, often acknowledging limited knowledge.

The difference is stark with medical questions. Ask "What's the treatment for X condition?" and you'll get authoritative-sounding advice that could be dangerously wrong. Ask "What's the treatment for X condition? If this is outside your reliable knowledge, please say so instead of speculating," and you're much more likely to get a disclaimer and a suggestion to consult an actual doctor.

Ask for Sources (and Watch the Confidence Crumble)

Request citations, links, or sources for any factual claims. This is the equivalent of asking to see the actual spreadsheet after they've quoted that revenue number.

Most AI chatbots will either retreat or start hedging when pressed for specifics. They might admit they can't provide links, or they'll offer vague sourcing like "according to recent studies" without naming which studies.

Say you're researching productivity statistics for a presentation. The AI tells you "Studies show that remote workers are 35% more productive than office workers." Ask "Which study? Can you give me the journal name and publication date?" Now you'll either get an admission that it can't cite a specific source, or you'll get something vague enough that you know to verify before putting it in your deck.

Make It Explain Its Reasoning Step by Step

Add "show your work" or "walk me through your reasoning" to your prompts. When required to articulate the logic chain instead of jumping straight to a conclusion, the AI often catches its own gaps.

The phrase "think step by step" has become almost magic in prompt engineering circles because it triggers more careful, hedged responses. The system has to slow down and examine each piece of reasoning, which surfaces uncertainty.

Ask "Is this plant safe for cats?" and you'll get a confident yes or no. Ask "Is this plant safe for cats? Please explain what you know about this plant's toxicity and walk me through how you reached that conclusion," and you're more likely to see phrases like "I believe" or "it's generally considered" or even "I should note I'm not certain."

This works beautifully for complex calculations or logical problems where you actually need to verify the reasoning, not just accept an answer.

Offer Multiple Choice With 'I Don't Know' as an Option

Structure your questions to include uncertainty as a legitimate answer. Instead of "What year did X happen?" try "Did X happen in 1995, 2003, or are you not sure?"

You've reduced the pressure to generate a plausible-sounding answer by making "I don't know" one of the acceptable responses. The system will take the exit door if you build one into the conference room.

This is perfect for fact-checking. You're writing an article and need to verify a claim. Instead of "Tell me about this obscure software feature," ask "Does this feature exist in version 2.0: yes, no, or are you not certain?" You'll get more honest responses because you've framed uncertainty as a valid option, not a failure.

Cross-Examine Like You're Deposing a Witness

Ask the same question multiple ways in sequence. Fabricated answers rarely stay consistent under repeated questioning.

Watch what happens when you probe from different angles. The story about Q3 numbers starts falling apart by the third explanation. Details shift. Confidence wavers. Hedging language appears ("it's possible," "generally," "typically").

Try this with any technical process or recipe where accuracy matters. Ask about a coding solution. Then ask "What are the potential problems with that approach?" Then "Would this work if the user is on a mobile device?" Then "What happens if the database is empty?" Each question forces the system to examine the answer from a new angle, and confidence tends to decrease with each round.

You're not trying to trick it. You're stress-testing the knowledge to see if it holds up or falls apart under scrutiny.

So what can YOU do with this?

When you're in research mode, make it standard practice to add "cite your sources and admit if you're uncertain" to every prompt about facts or current events. Treat it like a disclaimer you include automatically.

For anything you plan to publish or present, use the cross-examination technique. Ask the same thing three different ways and see if the answers stay consistent. If they don't, you know you're getting fabrication rather than knowledge.

Learning a new skill? Always request step-by-step reasoning in tutorials. "Explain how to do X" becomes "Explain how to do X, and walk me through your reasoning at each step." You'll catch gaps before you waste time following flawed instructions.

Before trusting AI advice on anything high-stakes (health, legal, financial matters), structure your questions with explicit uncertainty options. You want the system to feel safe saying it rather than making something up.

Build a daily habit: treat any confident-sounding answer about facts as a starting point for verification, not the final word. The more important the information, the more you verify. Look up those studies. Check those statistics. Confirm that restaurant is actually still in business.

TL;DR

• AI systems are trained to always sound confident, even when they're inventing information, because they predict plausible text patterns rather than retrieving facts
• Give explicit permission to say "I don't know" in your prompts, ask for sources, and request step-by-step reasoning to surface uncertainty
• Use multiple-choice questions that include uncertainty as an option, and cross-examine important claims by asking the same thing several different ways
• Verify anything that matters before trusting it, especially for research, publishing, or high-stakes decisions

At least when your colleague makes up revenue numbers, you can ask to see the spreadsheet. The AI will generate the spreadsheet too.