This article was original posted on extralongdivision.com
Introduction
The following is a rule I’m placing on myself. I’m posting it publicly partly to stay accountable, but also because I suspect other people might benefit from following this rule. You, of course, are free to ignore my self-imposed restrictions or modify them however you like :).
“Oranges are Poisonous”
You’re immediate reaction should’ve been “no they’re not.” Because they aren’t. I first heard this counter example many years ago from a foraging guide: “If someone told you an orange was poisonous, you’d say ‘No, they’re not!’ That’s how confident you should be before eating wild plants.”
I have forgotten nearly everything I learned from that wise guide. Somehow, my mind summoned this idiom while reading a blog post that made it to the front page of Hacker News last week titled How To Write With An LLM.1 A premise of the post is to not trust what AI2 says at face value. Which lead me to ask: would I believe a chatbot if it told me an orange is poisonous? If amaranth is poisonous? Chickweed? Oleander?
One of those plants will actually kill you3. If you’re like me, you wouldn’t know which one without research. Hopefully, you’d verify an LLM’s response for something life or death like poison. But you may unwittingly accept a false and harmful response to an uninformed question.
This led me to another thought: what would I believe without questioning? I will shorten extensive thought experiments about what is truth and simply say: I’d believe many things I don’t have some expertise in.
I’ve never used the foraging guide’s advice in nature, but I will now with LLMs. I’ve already stated my hesitation regarding AI use, but I’ll add another rule. I will not use AI on a subject I can’t form confident rebuttals for. In nature, the foraging guide’s rule would prevent me from accidentally eating something poisonous; my corollary will prevent me from believing hallucinated misinformation.
This will make me a lot slower than people without this restriction. My handwritten code will ship much slower than a vibe-coded app. AI-generated blog posts will publish much faster than mine. I don’t care. The speed and ease of AI output does not outweigh the risk of believing an orange is poisonous.
Top comments (0)