DEV Community

Michael Mitrakos
Michael Mitrakos

Posted on

How I stopped an LLM from lying on your resume (grounding, in practice)

#ai #llm #webdev #career

TL;DR: Telling an LLM "only use real information" in a system prompt is not a safeguard — it's a suggestion the model will ignore the moment a keyword filter rewards ignoring it. If you need a hard guarantee that generated text only makes sourceable claims, you have to build it outside the prompt: constrain generation to a fact store, then run a separate verification pass that rejects any claim it can't trace back to a source. Below is the pattern, with code.

I've been building an AI agent that writes and submits job applications (LandEarly — that's the context, not the point of this post). The single hardest requirement wasn't scraping job boards or automating Workday's form-from-hell. It was this:

The system must never claim the candidate has experience they don't have.

Sounds trivial. It is not. And the way most tools "solve" it — a stern line in the system prompt — is exactly why so many auto-apply tools produce resumes that fall apart in the first phone screen.

Let me walk through why the easy version fails and what actually works.

Why "just tell it not to" doesn't hold

Here's the naive approach almost everyone starts with:

const system = `
You are a resume writer. Tailor the candidate's resume to the job.
IMPORTANT: Only use real information. Never invent experience.
`;
Enter fullscreen mode Exit fullscreen mode

The problem: the model has two instructions in tension. "Match this job" pulls toward inventing a Kubernetes bullet when the JD screams Kubernetes. "Don't invent" pulls the other way. Which one wins is non-deterministic, and it gets worse under exactly the conditions you care about — a strong keyword match the candidate doesn't actually have is the most tempting thing for the model to fabricate.

You cannot audit a vibe. If the guarantee lives only in the prompt, you have no guarantee.

Step 1: Generation can only see facts, and every fact has an ID

The first move is to stop feeding the model a blob of resume text and start feeding it a structured, addressable set of facts.

// The candidate's real history, as discrete sourceable units.
const factStore = [
  { id: "exp_01", text: "Built a React dashboard used by ~2k internal users at Acme" },
  { id: "exp_02", text: "Migrated a REST API to GraphQL, cut payload size ~40%" },
  { id: "skill_01", text: "TypeScript — 4 years, primary language" },
  // ...
];
Enter fullscreen mode Exit fullscreen mode

Now generation is constrained: the model must write each resume bullet from a specific fact, and return the id it used.

const system = `
You tailor resumes. You may ONLY write bullets grounded in the FACTS provided.
For every bullet you output, return the exact id of the fact it is based on.
If no fact supports a requirement in the job description, DO NOT write a bullet
for it. Return JSON only:
{ "bullets": [{ "text": "...", "sourceId": "exp_01" }] }
`;

const res = await callModel({ system, facts: factStore, job: jobDescription });
const { bullets } = JSON.parse(res);
Enter fullscreen mode Exit fullscreen mode

This alone changes the shape of the problem. We've gone from "trust the model" to "the model made a claim about its sources that we can now check."

Step 2: Never trust the claimed source — verify it

The model will sometimes cite exp_01 for a bullet exp_01 doesn't actually support. So the citation isn't the guarantee either. It's just a claim to be verified.

Two layers here. First, a cheap structural check — does the cited id even exist?

const byId = new Map(factStore.map(f => [f.id, f]));

function structurallyValid(bullet) {
  return byId.has(bullet.sourceId);
}
Enter fullscreen mode Exit fullscreen mode

Then the real check: does the source fact entail the bullet? This is a separate, narrowly-scoped LLM call whose only job is entailment — no creativity, no tailoring, no incentive to please. Small model, temperature 0.

async function isEntailed(bullet) {
  const source = byId.get(bullet.sourceId);
  const prompt = `
SOURCE FACT: "${source.text}"
CLAIM: "${bullet.text}"
Does the SOURCE FACT fully support the CLAIM, with no added specifics
(numbers, tools, scope) that aren't in the source? Answer "yes" or "no".
`;
  const answer = await callSmallModel({ prompt, temperature: 0 });
  return answer.trim().toLowerCase().startsWith("yes");
}
Enter fullscreen mode Exit fullscreen mode

The key insight: the verifier is a different job than the generator. The generator is optimizing to match the JD, which is precisely the pressure that produces fabrication. The verifier has no knowledge of the job at all. It can't be tempted, because it doesn't know what the tempting answer would be.

Step 3: Reject, don't repair

The last piece is a policy decision, and it's the one people get wrong: when a bullet fails verification, you drop it. You don't ask the model to "fix" it, because "fix" quietly becomes "make it pass," which is fabrication with extra steps.

async function groundedBullets(candidate) {
  const drafted = await draftBullets(candidate);       // step 1
  const checked = [];

  for (const b of drafted) {
    if (!structurallyValid(b)) continue;               // fake id → drop
    if (!(await isEntailed(b))) continue;              // unsupported → drop
    checked.push({ ...b, source: byId.get(b.sourceId).text });
  }

  return checked; // every survivor is traceable to a real fact
}
Enter fullscreen mode Exit fullscreen mode

If that leaves the resume thin against a role, that's not a bug to paper over — it's signal. It means the candidate is a weak match for that requirement, and the honest product move is to surface that ("this role wants 5 years of Go; you have none") rather than fake it.

The part users actually see

Because every surviving bullet carries its source, the UI can show, next to each line, exactly which piece of the person's real history it came from. That turns "trust our AI" into "check our work" — the reviewer can see the receipts. It's also the single biggest trust feature we shipped, and it fell out of the architecture for free once grounding was in place.

What generalizes beyond resumes

Strip out the domain and this is a reusable pattern for any LLM feature where being wrong is expensive:

  1. Make the source material addressable. Discrete units with IDs, not a text blob.
  2. Force the generator to cite. Output must reference which unit it used.
  3. Verify with a separate, incentive-free pass. The checker must not share the generator's goal.
  4. On failure, drop — never auto-repair. Repair pressure is fabrication pressure.
  5. Expose the sources to the user. Traceability is a feature, not plumbing.

It's the same instinct behind good RAG, just pointed at self-consistency instead of retrieval accuracy. And it's a lot more work than one line in a system prompt — but "don't make things up" is a requirement you either build for or you don't actually have.

If you want to see the end result in the wild, it's what runs under LandEarly. But mostly I wanted to write up the pattern, because there's shockingly little honest writing about the gap between "the prompt says don't hallucinate" and "the system provably doesn't."

How are you handling this in your own LLM features? Especially curious how others approach the verifier step — separate model, same model with a fresh context, or something structured like constrained decoding. Tell me in the comments.

Top comments (0)