Don't Classify, Hallucinate: A Counterintuitive ML Approach That Actually Works

Don't Classify, Hallucinate: A Counterintuitive ML Approach That Actually Works

A blog post titled "Don't classify, hallucinate" by software engineer Doug Turnbull reached the Hacker News front page with 216 points and 85 comments. Its central argument challenges a fundamental assumption in machine learning: that classification is always the right approach.

The Core Insight

Traditional ML classification asks: "Given this input, which category does it belong to?" You train a model on labeled examples, and it learns to assign new inputs to categories.

The "hallucinate" approach asks something different: "Given this category, what would the input look like?" Instead of learning to classify, the model learns to generate examples of each category.

This is the difference between:

• Classification: "Is this image a cat or a dog?" → outputs "cat"
• Hallucination: "Generate what a cat looks like" → compare with input → "yes, this looks like what I'd generate for 'cat'"

Why This Works Better Than You'd Expect

The key insight is about information density. Classification models compress all their knowledge into a single decision boundary. Generative models, by contrast, learn the full distribution of each class.

This matters because:

1. Calibration: Classification models are notoriously overconfident. A softmax output of 0.99 doesn't mean 99% confidence — it means the model's decision boundary is on one side. Generative models give you a likelihood, which is a more honest assessment.

2. Out-of-distribution detection: A classifier will confidently assign a new input to one of its known classes even if the input is nothing like anything it's seen before. A generative model can say "this doesn't look like anything I'd generate" — which is a much more useful signal.

3. Few-shot learning: If you have one example of a new class, a classifier needs to be retrained. A generative model can often generalize from a single example by comparing it to what it would generate.

4. Interpretability: When a generative model says "this input is unlike what I'd generate for any class," you can ask it to generate what it would expect, giving you insight into its reasoning.

The Connection to LLMs

Large language models are already generative models. When you ask GPT to classify a text, it's actually generating a response that happens to be a classification. The "hallucinate" approach suggests we should lean into this rather than fighting it.

Instead of training a classifier to detect spam, you could:

1. Train a language model on spam and non-spam emails
2. For a new email, ask: "Would this email be more likely under the spam model or the non-spam model?"
3. Compare the likelihoods

This is essentially what likelihood-based anomaly detection does, and it's been shown to outperform classifiers in many settings, especially when the training data for one class is limited.

Practical Applications

Turnbull's post highlights several use cases where this approach shines:

Search relevance: Instead of training a classifier to predict "relevant" vs "not relevant," train a model to generate relevant results and measure how well actual results match. This is closer to how modern search engines actually work.

Recommendation systems: Instead of classifying "will user click" vs "won't click," model what a user would click on and compare.

Fraud detection: Instead of binary fraud classification, model what legitimate transactions look like and flag anything that doesn't match the distribution.

Content moderation: Instead of classifying content as "safe" or "unsafe," model what safe content looks like and flag deviations.

The Tradeoffs

This isn't a free lunch:

• Computational cost: Generative models are typically more expensive to run than classifiers
• Training data: You need enough data to model the full distribution, not just a decision boundary
• Evaluation: It's harder to evaluate generative models with standard metrics (accuracy, F1) since they output distributions, not labels
• Latency: For real-time applications, generating to classify adds inference time

The Bigger Lesson

The deepest lesson from this approach is about modeling vs. deciding. Classification is a decision problem: which bucket does this go in? Generation is a modeling problem: what does this distribution look like?

When you model the distribution, you get the classification for free (pick the class with the highest likelihood). But you also get everything else: anomaly detection, calibration, interpretability, and the ability to handle new classes.

In the era of LLMs, where every model is already a generative model, the "don't classify, hallucinate" approach isn't just a clever trick — it might be the natural way to use these models.

---

Based on Doug Turnbull's blog post (216 points on Hacker News).