The pleasing effect: why AI grades your CV on a curve

I recently wrote about the halo effect associated with AI: the idea that a model that is good at one thing must be good at everything. I presented a simple example in which even the smartest available model could not write a basic letter in German.

This time, I will discuss a different problem: the pleasing effect. AI tends to tell you what works, and that can be dangerous if the information it omits is important.

Imagine you are job hunting and have your CV in front of you.

What does everyone do? Write a careful prompt. Tell the AI to be critical and avoid flattery. Paste the CV and job description. Ask for the truth.

And it delivers. It identifies genuine weaknesses and provides specific arguments.

That is the trap. The AI decides what counts as a weakness after it has read your CV, and tends to pick the standards you happen to meet.

I tested this. The results were worse than I expected.

There is a solution: an additional prompt. First, show the model only the job advert. Then, ask what the hiring manager actually wants but never wrote down. Finally, hand that list back with the CV. It sounded like prompt engineering folklore. It isn't.

The setup

There are two job ads and three candidates for each. For the fintech backend role, I identified five obvious priorities for a hiring manager, none of which are mentioned in the ad: experience of carrying a pager for systems that move real money; ability to work with no PM or spec; experience of working somewhere with under 150 people; ability to still write code; and understanding of what happens when the ledger is wrong.

Priya meets all five criteria, but her skill set comprises Kotlin and NATS rather than the Go/Kubernetes/Kafka mentioned in the ad, and her job title is 'Member of Technical Staff'. To a keyword scan, she looks unremarkable.

Daniel is a perfect match in terms of job title and the technologies mentioned in the ad for the payments company. However, if you read closely, you will see that he has never been near the money path: his role is to report on settlements from data provided by another organisation, he is only on call during business hours, and he has spent two years leading rather than shipping.

Marcus is a front-end engineer. The control.

None of the CVs say any of this about their candidates. You have to infer it.

The two prompts:

Normal:

Here is a job description and a candidate's resume.

[job description]
[resume]

Give me a thorough analysis of how well this resume matches the job:
a fit score, a breakdown of strengths, and a gap analysis of what's
missing. Be rigorous and specific.

Two-step. First, with no CV in context:

Here is a job description.

[job description]

Do NOT evaluate any candidate. Reconstruct the IMPLICIT expectations
behind this posting — the things that are NOT written down but are
actually in the head of the hiring manager and whoever else signs off
on this hire. What would make someone succeed or fail here that the
ad never says out loud? Read between the lines: the wording, the
omissions, the ordering, the company stage, the "nice to haves."

Then build an evaluation scale:
- The unwritten must-haves, and why each matters to THIS org
- Which stated requirements are actually soft, and which stated
  "nice to haves" are actually hard requirements
- Disqualifiers that would never appear in the posting
- For each dimension: what a VERY GOOD match looks like, what a GOOD
  match looks like, and what a BAD match looks like — in terms of
  evidence you could actually find on a resume
- Relative weighting across dimensions

Then:

[job description]
[the scale from step one]
[resume]

Place this candidate on the scale. Go dimension by dimension and cite
the specific resume evidence for each placement. Where the scale
asserts a criterion the job description doesn't support, say so and
discount it.

What came back?

From the job advertisement alone, Step One identified all five hidden priorities. Regarding the compliance one, which the ad had filed under 'nice to have', it wrote:*

Candidate	Truth	Normal	Two-step
Priya	real fit	87 — interview	96 — interview
Daniel	fake fit	70 — interview	65 — maybe
Marcus	wrong	22 — reject	40 — reject

Either way is fine. Both got the order right.

Then the marketing job.

Candidate	Truth	Normal	Two-step
Alessandra	real fit	74 — interview	79 — interview
Tyler	fake fit	76 — interview	67 — maybe
Rachel	wrong	29 — reject	8 — reject

The normal way scored the fake candidate above as the real one and sent them both to interview.

Why does it fail exactly?

This is the pleasing effect, and it doesn't look like flattery.

Tyler was familiar with the normal way. It identified most of his issues and even stated that his keyword match 'should be discounted'. There is nothing soft about the output.

The failure is more subtle than being fooled. When the model reads a CV and a job advert together, it invents whichever standard makes the person in front of it look good.

When reading Alessandra's CV, it wrote:

'She measures adoption, which almost no PMM candidate does.'

She had found that 71% of enterprise representatives had actually used her materials, up from 19%.

Then it read 'Tyler'. His CV states that he published 38 assets in 18 months and makes no mention of whether any of them were opened by a single representative. The adoption test never came up. It had been introduced for one candidate earlier and then quietly dropped. Instead:

The figure of 38 assets in 18 months demonstrates sustained throughput.
This is the actual failure mode of enablement work.

The same absence. It turned into a strength.

The standard changes depending on who is in front of it. This is why interviewers write their questions before meeting anyone.

The obvious objection:

'That's not about deriving hidden expectations. You gave one version a structured rubric and the other a vague 'Give me a score.' Of course, the structured one is more consistent.'

I ran it as a single pass with the job ad and CV together, with no derivation step and forced onto an identical structured scale.

Evaluate this candidate in a SINGLE pass, using this exact procedure:
1. Choose 8-12 evaluation dimensions.
2. Weight each dimension; weights must sum to exactly 100.
3. Score each on an anchored 0-4 scale: 0 = no evidence, 1 = weak,
   2 = partial, 3 = good, 4 = very good.
4. Compute the weighted sum and show the arithmetic.
   Do not adjust the number afterward for any reason.

It rated the fake candidate 84 and the real one 78. It was backwards again, wider than before.

The scale wasn't doing the work. It was the order of operations that was doing the work.

Version	Derives first?	Job 1	Job 2	Backwards
Normal	no	+17	−2	1 of 2
Two-step, rubric only	yes	+31	+12	0 of 2
Two-step, keeps job ad	yes	+30	+12	0 of 2
Structured, one pass	no	+15	−6	1 of 2

(Real candidate's score minus fake candidate's; positive means correct order)

One change to the recipe:

Don't hide the job ad in step two. In the original version of this trick, only the derived scale is passed to the scorer. Showing both the ad and the scale produced identical results in every cell, and hiding the ad comes at a cost. One generated scale created an inflexible rule that rejected anyone not living in the right city, despite this not being stated in the ad. This rule was applied to one candidate and quietly waived for another despite identical circumstances. If applied consistently, it would have automatically rejected the two-step's own best candidate. A scorer who can't see the ad can't identify that issue.

So, what s

One: job ad only, no CV. "What does the hiring manager actually want that isn't written here? What does a very good, good and bad match look like for each of these?'

Two: job ad + answer to the above question + CV. "Where does this person land? Discount anything in the list that the ad doesn't support.'

One extra prompt. If you're the one applying, step one alone is useful — it tells you what the advert is really asking for, which is better than any score for guiding you to rewrite your CV.

The honest caveat:

There were four head-to-head comparisons, with two job ads and one run each. The direction was the same all four times and the mechanism was clear, but this was a small test and not a measured effect size. The CVs were synthetic and designed so that surface polish and real fit pulled in opposite directions — an interesting case and one that is most favourable to the finding. I'd expect the gain to shrink with a vague job advert. There's less subtext to recover when there isn't any.
polish: improved 50 of 50 lines.