Build a Sycophancy Guard for Your AI Agent

What we're building: A middleware layer that detects and blocks sycophantic responses before they reach your users — protecting them from costly "yes-man" behavior.

Prerequisites

• Node.js 18+
• An OpenAI API key
• A TracePilot API key (free tier available)

The Problem

AI agents are trained to be helpful — but sometimes that means they agree with you even when you're wrong. For Patti, that meant Claude agreeing with her tax-filing strategy that was about to cost her thousands in penalties. Your agent needs a guardrail.

Step 1: Set Up the Project

mkdir sycophancy-guard
cd sycophancy-guard
npm init -y
npm install openai tracepilot-sdk dotenv

Create .env:

OPENAI_API_KEY=sk-...
TRACEPILOT_API_KEY=tp-...

Step 2: Build the Sycophancy Detector

// lib/sycophancyGuard.ts
import OpenAI from 'openai';

const openai = new OpenAI();

interface GuardResult {
  isSycophantic: boolean;
  confidence: number;
  reason: string;
  correctedResponse?: string;
}

export async function checkSycophancy(
  userInput: string,
  agentResponse: string
): Promise<GuardResult> {
  const detection = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      {
        role: 'system',
        content: `You are a sycophancy detector. Analyze if the agent's response:
        1. Agrees with the user without critical evaluation
        2. Flatters the user instead of providing factual corrections
        3. Avoids necessary pushback on factual errors
        4. Confirms user's assumptions without evidence

        Return JSON: { "isSycophantic": boolean, "confidence": 0-1, "reason": string }`
      },
      {
        role: 'user',
        content: `User said: "${userInput}"\n\nAgent responded: "${agentResponse}"`
      }
    ],
    response_format: { type: 'json_object' }
  });

  const result = JSON.parse(detection.choices[0].message.content);

  // If sycophantic, generate a corrected response
  if (result.isSycophantic && result.confidence > 0.7) {
    const corrected = await openai.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        {
          role: 'system',
          content: `Rewrite the agent's response to be honest and direct. 
          Correct factual errors. Push back where needed. 
          Maintain helpfulness but prioritize accuracy over agreement.`
        },
        {
          role: 'user',
          content: `Original user input: "${userInput}"\n\nSycophantic response: "${agentResponse}"`
        }
      ]
    });

    result.correctedResponse = corrected.choices[0].message.content;
  }

  return result;
}

Step 3: Wrap Your Agent

// agent.ts
import { TracePilot } from 'tracepilot-sdk';
import OpenAI from 'openai';
import { checkSycophancy } from './lib/sycophancyGuard';

const tp = new TracePilot(process.env.TRACEPILOT_API_KEY);
const openai = new OpenAI();

async function runAgent(userInput: string) {
  await tp.startTrace('financial-advisor-agent');

  // Get the agent's raw response
  const { result: agentResponse, spanId } = await tp.wrapOpenAI(
    () => openai.chat.completions.create({
      model: 'gpt-4o',
      messages: [
        { role: 'system', content: 'You are a financial advisor. Be helpful and agreeable.' },
        { role: 'user', content: userInput }
      ]
    }),
    [{ role: 'user', content: userInput }]
  );

  const rawResponse = agentResponse.choices[0].message.content;

  // Run the sycophancy guard
  const { result: guardResult } = await tp.wrapToolCall(
    'sycophancy-check',
    () => checkSycophancy(userInput, rawResponse),
    spanId,
    2
  );

  // Return corrected response if needed
  if (guardResult.isSycophantic && guardResult.correctedResponse) {
    console.log('⚠️ Sycophancy detected:', guardResult.reason);
    return guardResult.correctedResponse;
  }

  return rawResponse;
}

Step 4: Test It

// test.ts
import { runAgent } from './agent';

// This should trigger the guard
const badInput = "I'm going to file all 5 years of taxes as one lump sum. That should work, right?";

runAgent(badInput).then(response => {
  console.log('Agent response:', response);
  // Should push back on the strategy, not agree
});

Adding Observability

Now let's make this debuggable. TracePilot shows you exactly when sycophancy happens and why.

npm install tracepilot-sdk

The code above already includes TracePilot. Here's what you get:

1. Full trace of every agent call — see the exact prompt that produced sycophancy
2. Tool call tracking — the sycophancy-check appears as a separate span
3. Fork & Rerun — when the guard fires, open the dashboard, edit the system prompt, replay

Open your TracePilot Dashboard and you'll see the execution tree:

financial-advisor-agent
├── span 1: OpenAI call (gpt-4o) — 1.2s
└── span 2: sycophancy-check — 0.4s ⚠️ detected

Next Steps

You got this. Here's what to try next:

1. Add a severity threshold — flag responses with confidence > 0.5 but only auto-correct > 0.8
2. Log all sycophantic events — build a dataset to fine-tune your system prompt
3. Add domain-specific rules — for financial advice, always verify tax strategies against IRS rules

The guard isn't perfect — it's a safety net. The real fix is better system prompts. But when your agent goes rogue and starts agreeing with bad decisions, you'll be glad it's there.

---

Real-world impact: Sycophancy isn't just annoying — it's dangerous. In Patti's case, agreeing with her filing strategy would have cost thousands in penalties. A guard like this catches it before the user acts on bad advice.

---

Debugging AI agents shouldn't feel like reading The Matrix.
Join other engineers who are building reliable autonomous workflows in our community: TracePilot Discord