Ask an LLM to summarise a contract and it does a reasonable job. Ask it to give you a compliance score you'll act on, and run the same document through it twice, and you may get two different scores. That inconsistency is fine for a summary and unacceptable for anything downstream of it.
Separate extraction from scoring
In DocuLens AI, the LLM's job stops at extracting structured fields — vendor, amount, date, line items. A schema-validated, deterministic layer then scores those fields against defined rules. The LLM's variability is contained to the part of the pipeline that can tolerate it.
Schema validation is not optional
If the extracted output doesn't match a defined schema, reject it and retry or flag it — don't let malformed output silently propagate into a compliance score someone will act on.
The rule of thumb
Let the LLM read. Let deterministic code decide. The moment a decision needs to be reproducible and explainable, it shouldn't be resting on a single LLM call's non-deterministic output.
Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to
Top comments (0)