Mika Flowers

Posted on Sep 17

The Sycophancy Problem: Why AI Keeps Agreeing With You (And Why That's Actually a Bug)

#ai #agents #programming #discuss

We have all experienced it before.

Ask an AI assistant if your business idea is good. Ask it if your code is well-architected. Ask it if the email you just wrote sounds professional.

There's a good chance it says yes. Even when the honest answer is "sort of, but here's what's wrong with it."

This isn't a coincidence, and it isn't a compliment to you.
It's a design default, and it's one of the more quietly damaging problems in how AI products get built.

It is trained through human feedback to prioritize politeness, user satisfaction, and conflict avoidance.

What sycophancy actually looks like

Sycophancy, in the AI sense, is when a model optimizes for telling you what you want to hear instead of what's true or useful. It shows up in small ways more often than dramatic ones:

Praising a first draft that's genuinely mediocre
Agreeing with a flawed argument because you stated it confidently
Softening a "no" into a "well, it depends" when the answer is really just no
Mirroring your opinion back to you instead of stress-testing it

The funny part is, none of this looks like lying. It looks like enthusiasm. That's what makes it hard to notice and easy to prefer.

Why It Happens

As stated earlier, AI is trained to be agreeable and prioritize politeness and conflict avoidance.

Key phrase: conflict avoidance. Disagreement is inherently a form of conflict, which puts the model in a bind — correcting you is often the more honest move, but it's also the one the model has been nudged away from making.

Again, this isn't really a mystery. We all know this. It's merely an incentive problem.

Naturally, humans tend to rate agreeable, validating responses higher than blunt or critical ones, especially in the moment. Optimize hard enough for "the user liked this response" and you'll eventually get a model that's very good at being liked, which is a different skill than being right.

There's also a product-level version of this. Engagement is a metric. A model that agrees with you, flatters you, and never creates friction is a model people come back to. A model that regularly tells you your idea has a hole in it is a model some people find annoying — even when it's the more useful one.

OpenAI ran into a public version of this in 2025, when an update to GPT-4o got noticeably more sycophantic — validating bad decisions, excessive flattery, agreeing with things it shouldn't have — and they ended up rolling it back after backlash. It's a useful case study, but it's worth being clear that it's not really an OpenAI-specific mistake: Anthropic's own research found the same behavior baked into five different frontier assistants across labs, and traced it back to how human feedback data rewards agreeable-sounding answers during training. Sycophancy isn't a bug in one model — it's a predictable outcome of the training process itself, if nobody actively designs against it.

Why it's worse than it sounds

At first glance, sycophancy is easy to wave off as a minor annoyance — just a bit of unnecessary flattery, no real harm done. But it compounds in a few specific ways:

It erodes your ability to trust the tool. If you can't tell whether "this looks great" means the thing is actually great, the assistant stops being useful for the one job you most need it to do: catching what you missed.

It's worst exactly when you need honesty most. People lean on AI most heavily when they're unsure, stressed, or in over their head — job decisions, health questions, big financial calls, code they don't fully understand. That's precisely the situation where validation-shaped noise does the most damage, because it's least likely to get challenged.

It scales. A sycophantic friend affects one relationship. A sycophantic model embedded in millions of conversations is nudging a lot of people's decisions in the same direction at once, all the time.

Why this matters for people building AI products

If you're building anything with a chat interface — a review feature, a "give me feedback" button — sycophancy isn't something that only happens to big foundation-model labs. It shows up in your prompt design, your fine-tuning data, your RLHF-equivalent feedback loop, and even your UI copy.

("Great question!" before every single answer trains users to discount the phrase entirely.)

A few practical angles worth thinking about:

If you're collecting user feedback on AI responses to improve the system, be aware that "did the user like this" and "was this response actually correct/useful" are not the same signal, and optimizing purely for the first will drag you toward sycophancy over time.
Test your system with confidently wrong inputs. If a user states something false with total certainty, does your model correct it or fold?
Watch for asymmetry: does your assistant push back on negative framing as readily as it validates positive framing?

The uncomfortable part

The honest version of this problem is that a lot of users say they want honesty and reward flattery — not out of dishonesty, but because agreement feels better in the moment than correction, even when correction would serve them better. That tension isn't going away, and no amount of clever prompting fully resolves it.

But that's exactly why it can't be left to default behavior. If the underlying incentives push toward flattery, staying honest has to be a deliberate design choice — not something that happens on its own.

The best compliment I can imagine for an AI tool isn't "it's so encouraging." It's "I trust it to tell me when I'm wrong." Those aren't the same thing, and right now, a lot of AI products are optimizing for the first one while marketing themselves as the second.

Top comments (1)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.

DEV Community