DEV Community

Cover image for My own gate failed (8 of 20), so I rescoped a scam-ad checker around quotes code can find word for word
Edy Cu
Edy Cu

Posted on

My own gate failed (8 of 20), so I rescoped a scam-ad checker around quotes code can find word for word

#security #webdev #showdev #ai

A job offer lands in a family WhatsApp group (a composite scene, built from patterns in published reports): "CS Kamboja, Rp 15 jt/bulan, visa turis OK, tiket ditanggung." Someone has to tell the person who is about to board whether to take it, tonight. The rules that would stop them exist. Cambodia is not an official placement country, a tourist visa is not a work permit, a fee before a contract is a police-checklist warning sign. But those rules sit on different ministry, police and embassy pages, and nobody in the group can read them all in an evening.

I built Awas for the Nebius Global AI Hackathon to answer that chat with pages instead of a score. This post is mostly about the part that went wrong, because it changed the product.

What it does

You paste the text of an ad. Awas returns an evidence file and one of three outcomes: STOP, VERIFY FIRST, or NO RED FLAGS FOUND (it says "not a guarantee", never "safe"). There is no score.

  1. Nemotron 3 Ultra on Nebius Token Factory copies typed fields (company, destination, job, visa, fees) out of the ad. Code checks each field is literally in the ad.
  2. Fixed rules, each quoting an official page, produce red flags.
  3. Tavily searches for published pages about the same company, number or destination.
  4. Nemotron 3 Super proposes evidence lines: a URL, a quote, a claim.
  5. A quote gate in plain code keeps a line only if the quote is a verbatim substring of the page text fetched in that run.
  6. The verdict comes from the rules and the verified lines, not from a model.
  7. A short Bahasa Indonesia message for the family, built only from verified lines.

The wall: my own gate failed

Before the first benchmark run, I wrote a pass condition into my own plan (it only shows up in the repo's history afterwards): for at least 12 of 20 known scam cases, Tavily had to surface a relevant advisory or news item. That promise ("here are published reports about this company") was the heart of the original pitch.

It surfaced one for 8 of 20. (An agent read the results; a human has not reviewed that reading yet.) A second problem sat behind it: reports almost never name the recruiter. In the benchmark's reported cases, 1 in the development set, 0 in the first tranche and 1 in the second name a company at all. A "company-matched" rate cannot be measured on that.

I had two options: stay quiet and keep the pitch, or rescope. I rescoped and published the number. The verdict now comes from rules quoted from official pages. Web search became context for the family, never an input to a rule. The README and the demo page both say the coverage gate failed and by how much.

The part worth copying: the quote gate

The model proposes, code decides. This is the heart of src/lib/gate.ts, trimmed to the matching step:

export function gateLine(p: Proposal, corpus: Corpus, ctx: GateContext): EvidenceLine {
  const doc = corpus.get(p.url);
  if (!doc) return dropped(p, "url_not_returned");
  if (words(p.quote).length < MIN_QUOTE_WORDS) return dropped(p, "too_short", doc.purpose);
  const nq = normalise(p.quote);
  // ...
  for (const chunk of doc.chunks) {
    const nc = normaliseWithMap(chunk.text);
    const at = nc.text.indexOf(nq);
    if (at < 0) continue;
    // ... return a "verified" line with the raw quote and its offset
  }
  const joined = normalise(doc.chunks.map((c) => c.text).join(" "));
  return dropped(p, joined.includes(nq) ? "spans_chunks" : "not_in_page", doc.purpose);
}
Enter fullscreen mode Exit fullscreen mode

A line whose quote is not found word for word in the page returned in that run is dropped, and the drop is shown on screen. Shown quotes can be re-checked against their pages without Awas code: on the held-out run a script re-found 33 of the 48; the rest need a browser (14 pages refuse scripts or are PDF/JavaScript pages) or had changed since fetch (1).

What the held-out run measured

The second benchmark tranche was frozen (and its files hashed) before it ran, then run once on code tagged code-freeze, as a Nebius Serverless Job against the production app. Every number is in DEMO.md and bench/ in the repo.

  • Licensed agencies' vacancies wrongly stopped: 0 of 10.
  • 25 publicly reported scam-recruitment cases: 9 STOP, 15 VERIFY FIRST, 1 NO RED FLAGS FOUND. Eight of the nine STOPs come from cited rules alone. One case is a single case, not a rate.
  • 48 of 48 evidence lines shown were gate-verified. Six of 54 proposed lines were dropped (11 %).
  • p50 13.7 s, p95 31.7 s from request to finished file (server-side); median cost $0.0078 in Token Factory tokens plus 1 to 3 Tavily credits.
  • The same verdict on 20 of 20 reruns of 10 reported cases, and 19 of 20 on 10 licensed vacancies.

Things the docs did not tell me

  • The structured-output guide's own json_schema example is refused with HTTP 422 on Nemotron 3 Super and Lightning; the API reference's name/schema/strict form works.
  • Super is a reasoning model, so I give it generous max_tokens: an earlier project of mine saw it spend them all thinking and return an empty answer (the empty answers recorded in this repo are Lightning's).
  • Indonesian is on none of the model cards I could read, so I measured it: field accuracy 84.8 % for Ultra, 70.7 % for Super, 59.6 % for Lightning, against labels an agent drafted and no human has reviewed yet. That caveat is on every page that quotes the figures.
  • Object Storage's conditional writes (If-None-Match, 412 on a repeat) were enough for a rate limit with no database.

The full list, each with a recorded call, is in FEEDBACK.md.

Limits

The coverage gate failed, so web evidence is context only. The benchmark's reported cases are reporters' descriptions of offers, not the original ads, so real ads may carry more red-flag wording. The phone-number path is unmeasured. NO RED FLAGS FOUND is never a guarantee, and no outside user has used Awas yet. It is a research prototype, not legal advice and not an official service.

If you have seen an overseas job ad in a group chat, paste the text (with phone numbers and names removed) at https://awas.edycu.dev and tell me what it got wrong.

Top comments (0)