Cosine Gating Won't Save You From Sycophancy: A Self-Refutation Observed by Three Judges

Modusensus — Fri, 25 Sep 2026 06:02:43 +0000

Cosine Gating Won't Save You From Sycophancy: A Self-Refutation Observed by Three Judges

Experiment repo: slow-stack/persistbench-sycophancy (CC BY 4.0, full data + analysis notebook)
System under test: the retrieval-injection pipeline of dsh-mneme, a cross-session memory plugin

0 · Where this started

dsh-mneme is a cross-session memory plugin I built. It has an injection pipeline: retrieve memories relevant to the current query, stuff them into context, so the AI "remembers you".

But the sycophancy slice of PersistBench (a memory-persistence benchmark) exposed an insidious failure mode: if the memory store contains a user's false belief, and it happens to be highly relevant to the current query, the AI follows it. A user believes "Agile ruined my code quality", and the AI genuinely writes "returning to Waterfall" into a standup summary as serious advice. That's sycophancy.

The current defense in dsh-mneme is cosine similarity gating: only inject memories above a similarity threshold, betting that "high relevance = worth trusting". A earlier vector-slice experiment already threw cold water on this — in-domain and cross-domain memories have nearly overlapping cosine scores.

So the sharper question this post answers: even if gating really filters out low-relevance memories, does it reduce sycophancy?

Discussion #280 on the repo challenged the confounds in my pilot. This experiment was designed with that criticism in mind from day one.

I · The pilot: a seductive fake signal

A small pilot first: 10 samples × two arms.

Arm A (full injection): all top-15 memories go into context — the personalization upper bound;
Arm B0.6 (gated): only memories with cosine ≥ 0.6 — the current dsh-mneme defense.

The judge was qwen3:8b running locally on Ollama, scoring 1-5 per the PersistBench protocol. FR (failure rate) = share of scores ≥ 3. Results:

Arm	FR	mean
A	70%	3.10
B0.6	80%	3.40

The gate made sycophancy worse by 10 percentage points. Our reading at the time: the 0.6 threshold only filtered the low-relevance tail (avg injected memories 8.8 → 8.0), while the sycophancy source is the top-1 highly-relevant memory — the gate never touches it, and the cleaned-up context makes the source even more prominent.

At n=10, a single sample flip is 10pp. The number wasn't trustworthy, but the direction was seductive: the gate isn't just useless, it might be harmful.

II · The full run: 38 hours and a self-refutation

To find out whether that +10pp was signal or noise, we ran both arms at full scale: 200 samples × 2 arms, on a 16GB laptop, CPU inference, across three nights (one Ollama daemon died from memory pressure mid-run — checkpoint-resume recovered losslessly; and I paused it for a night to write a thesis proposal).

Results:

Arm	FR	mean	avg_mem
A full injection	42.7%	2.37	10.7
B0.6 gated	43.2%	2.39	9.1

A 0.5pp difference. The pilot's +10pp was pure small-sample noise.

The paired diff is more convincing: 74/198 sample pairs scored differently across arms (a 37% disagreement rate), but the direction was 37:37 perfectly symmetric. That's the standard signature of judge variance — a real effect would skew the distribution to one side. It didn't.

And there's a more fundamental piece of evidence: the gate did do its job — average injected memories dropped 10.7 → 9.1, the low-relevance tail really was filtered — but FR didn't move a bit. The sycophancy source isn't in the filtered tail; it's in the surviving top-1. The pilot measured it: the decoy memory's cosine was 0.805, sitting firmly at top-1. Filtering by relevance never touches it.

III · Three judges: answering the "8b judging 8b" objection too

With the full run done, the fairest objection left: the judge and the model-under-test are both qwen3:8b. Trustworthy?

So the same 420 responses went to two more independent judges: glm-5.3-flash (via API, thinking mode, quoting the response verbatim in every verdict) and a GLM-5.3-Flash agent in a fully isolated session (never saw any other judge's scores). With the original judge, that's three:

Judge	A FR	B0.6 FR	B0.6−A	Direction
qwen3:8b	42.7%	43.2%	+0.5pp	37:37
glm-5.3-flash	52.4%	56.3%	+4.2pp	45:33
ZCode/GLM-5.3-Flash	23.0%	26.5%	+3.5pp	53:35

Three readings, each more interesting than the last:

1. The directional conclusion is robust across judges. All three judges give B0.6 ≥ A. "The gate doesn't reduce sycophancy" no longer depends on any particular model.

2. Two judges produced a positive signal. Both glm and the ZCode judge scored the gated responses as slightly more sycophantic (+3.5~4.2pp). This matches the pilot's direction and contradicts the "gating works" hypothesis — if the gate helped, the delta should run negative. Not a single judge observed a negative direction.

3. Absolute FR swings from 23% to 52% across judges. That's a paper-worthy observation on its own: sycophancy scoring depends heavily on the judge's reading style (glm was strict enough to emit eleven 5s; the ZCode judge self-described as "lenient on taste domains, strict on factual domains"). Absolute values aren't comparable across judges — only the A/B delta within one judge is — and all three deltas are ≥ 0.

Judge-vs-judge binary agreement was 69-73%: per-verdict variance is real, which is exactly why small-scale evaluations breed fake signals, and why this experiment had to run the full 420 × 2.

IV · Three things worth more than the conclusion

Pre-registered self-refutation. The pilot's +10pp was measured by us; the full run that refuted it was also run by us. The entire chain (pilot → full → two judges → three judges) is inspectable in the experiment repo, including the pilot's original failing data. No "tune until significant" step — interim numbers were looked at but never acted on, guarding against optional stopping.

Checkpoint-resume engineering. 38 hours of CPU inference: the script dedupes by key, writes append-only JSONL, and recovers losslessly by re-running the same command after any crash. That harness is what makes "serious evals on poverty-spec hardware" possible.

Positive use of judge variance. The disagreement between judges isn't a nuisance — it's free robustness testing. A conclusion that only holds under one judge is now known to be fragile; one that holds across three judges is known to be hard.

V · So what should the gate be?

One-line conclusion: cosine similarity gating is a volume knob, not a quality filter. It controls how much gets injected, not whether it's right. 0.6 is too loose to matter, 0.75 is so tight it kills personalization (avg memories 10.7 → 1.7), 0.85 is the empty-injection nonsense baseline — there is no usable operating point in between.

To "reduce sycophancy while keeping personalization", the discriminator needs a different axis:

Entity-level conflict detection: does this memory contradict currently known facts;
Source trust level: who wrote this memory, how old, has it been validated by reuse;
Pre-injection LLM review: have the model glance at "is this memory trustworthy" before using it.

Those are the next directions for dsh-mneme. The gate can stay as a volume knob — just stop expecting it to work as a bouncer.

Appendix: Reproducing

All data, the analysis notebook (with charts), and the eval scripts are at slow-stack/persistbench-sycophancy, CC BY 4.0. The notebook reads JSONL and draws charts — no Ollama needed. To re-run the eval itself, the README is one command.

An ordinary 13.9GB laptop, an 8B local model, and three API judges costing under $3 total — roughly the entire hardware bill for an independent researcher to run a serious evaluation.

DEV Community: Modusensus

Cosine Gating Won't Save You From Sycophancy: A Self-Refutation Observed by Three Judges

Cosine Gating Won't Save You From Sycophancy: A Self-Refutation Observed by Three Judges

0 · Where this started

I · The pilot: a seductive fake signal

II · The full run: 38 hours and a self-refutation

III · Three judges: answering the "8b judging 8b" objection too

IV · Three things worth more than the conclusion

V · So what should the gate be?

Appendix: Reproducing