Quick answer: we tested the four safety checks in our own open-source Romanian question-answering demo against a model that was allowed to invent a...
For further actions, you may consider blocking this person and/or reporting abuse
The swapped deadlines are a good reason to label support at the relation level: actor, obligation, deadline and trigger belong together. Finding the number and overlapping words only establishes that the pieces exist, not that the document connects them as the answer claims.
I would extend the paired set with an answer that keeps the correct number but swaps the responsible actor, plus a supported paraphrase with low lexical overlap. A second-pass checker should reject the first and accept the second. Holding the generation prompt fixed would then let you measure the checker separately from changes in the model's own refusal behavior.
Thank you, this is the right way to cut it. Your swapped-actor case already happened once in this set without us naming it: the model answered 15 days to "how long does the applicant have to complain", and 15 days is in the text, but it is the institution's deadline to reply. Number correct, actor wrong. Labelling support at the relation level would have caught that; lexical overlap never will.
The low-overlap paraphrase is the half we had not built. Gate D would probably refuse it, so it measures the false-refusal side, which a set made only of traps cannot see.
And fixing the generation prompt answers the weakness we admitted in the article: our 4 vs 6 mixes the prompt change with the gates. Same prompt, gates on and off, is the clean comparison. If we run the extended set we will publish the result in the same repository, failures included.
The institution's 15-day reply deadline is a particularly useful fixture because every surface token can be correct while the obligation is assigned to the wrong party. I would retain the question, the actor-bound source span, and the expected refusal or correction together.
For the fixed-prompt comparison, report accepted wrong-actor answers and refused valid paraphrases separately. A stricter gate can reduce the first by increasing the second, so a single pass count would still conceal the tradeoff your extended set is meant to expose.
Agreed on both. One pass count would hide exactly the trade we are trying to measure, so the results will show two numbers: wrong-actor answers accepted and valid paraphrases refused. The 15-day case goes in as a fixture with the question, the actor-bound span and the expected correction kept together.
Keeping the actor-bound span with the 15-day question will make that fixture interpretable without the rest of the discussion. I would include a minimally changed answer that preserves the deadline but assigns it to the other actor, beside a valid paraphrase.
Those paired examples can share the same source text, making the actor binding the intended difference. Report both error counts with their separate denominators, so a check that rejects everything cannot look improved merely because wrong-actor acceptance falls to zero.
Good point on the denominators. Wrong-actor answers accepted out of all wrong-actor cases, valid paraphrases refused out of all valid paraphrases, so a gate that refuses everything shows up as 100% on the second number instead of looking clean. And yes, each pair will share the same source span, so the actor is the only thing that changes. Thank you for working through this with us.
Đúng vấn đề cốt lõi: citation faithfulness ≠ answer correctness. Các guardrail kiểu "verify provenance" chỉ bắt được unfaithful citation (trích dẫn sai nguồn), chứ không bắt được unfaithful generation (sinh ra thông tin sai khi đã không có nguồn).
Thực tế RAG evaluation cần tách biệt 3 lớp:
Bài test của bạn cho thấy safety checks chỉ cover lớp 2. Lớp 3 mới là nơi hallucination thực sự gây hại — model tự tin kết luận sai dù đã cite đúng (hoặc không cite vì evidence bị remove).
Một pattern thực tế: thêm answer-level verification — dùng một model nhỏ (hoặc rule-based) để check "does this conclusion logically follow from the cited evidence?" thay vì chỉ check "are citations present". Cái này bắt được cả trường hợp evidence bị remove và model vẫn hallucinate — found it via LabAgent, site: labagent .tech