The Problem We Couldn't Ignore
A buyer files a dispute: "Item not as described. The bag in the photo was leather. What arrived is plastic." The seller responds: "Item matches the listing exactly. Buyer is lying."
Two conflicting narratives. One order record. One blurry product photo. Somewhere in there is the truth — but no human agent will read it carefully at 11 PM on a Friday.
Online marketplaces collectively handle hundreds of millions of disputes every year. The resolution process at most platforms is, charitably, a keyword filter with a confidence score hardcoded to sound decisive. The systems that call themselves "AI-powered" are almost always rule engines that optimize for throughput over correctness. On genuinely ambiguous cases, they guess.
We kept asking: what would it look like if the AI actually reasoned over a dispute the way a careful adjudicator would? And — critically — what would it look like if it knew when to stop and say "I'm not sure"?
That question led us to Roman law.
The Latin Phrase That Named the Project
In ancient Roman courts, when a jury had heard all the evidence and still could not decide, a juror was entitled to return a verdict of non liquet — "it is not clear" — rather than force a guess. That option, the structured right to abstain, is what separates honest adjudication from fast adjudication.
The moment we articulated that principle, the project had a name and a north star: LIQUET — build an agent that resolves disputes it can resolve, and says so clearly when it cannot.
Every existing dispute-automation tool we examined had the same flaw: no principled abstention mechanism. The most dangerous failure mode in automated dispute resolution isn't slow resolution — it's a system that is wrong with high confidence.
What We Built
Liquet is a fully autonomous dispute arbitration agent. A dispute enters the system — through email, API, or the web dashboard — and Liquet runs a complete three-stage AI pipeline, stress-tests its own verdict with a built-in adversarial pass, runs adjudication three times to verify the reasoning is stable, and then makes a binary decision:
- LIQUET ✅ — confidence ≥ 80%, order value < $500, no hard contradictions, verdict stable → resolved autonomously, both parties notified, webhook fired
- NON LIQUET ⚠ — any condition violated → escalated to human reviewer with a complete decision brief
The gate is entirely deterministic Python. It cannot be hallucinated past.
The Three-Model Pipeline
One of the earliest and most important decisions we made was: don't route everything through one model.
Stage 1: qwen3.6-flash — Triage
Before spending a single premium token on full investigation, every incoming dispute goes through a fast triage pass. In milliseconds, qwen3.6-flash classifies the dispute category, scores its complexity, assigns a priority, and estimates resolution time. This pre-classification routes simple cases efficiently and prevents the expensive pipeline from wasting compute on disputes that can be resolved with minimal evidence.
Stage 2: qwen3.6-plus — Visual Evidence
Many marketplace disputes live or die on physical evidence: the product photo in the listing versus the photo the buyer uploaded. qwen3.6-plus handles every call that carries images — damage detection, label verification, packaging inspection, screenshot analysis of tracking pages. The vision model doesn't write verdicts; it writes evidence summaries that the reasoning model then weighs.
Stage 3: qwen3.7-max — Reasoning and Adjudication
The flagship model handles the calls where extended chain-of-thought actually improves correctness: the full adjudication pass over the assembled case file, the adversarial skeptic rebuttal, the email parsing that turns a buyer's complaint email into a structured case. qwen3.7-max also powers ReasoningGlass — a live stream of the model's extended-thinking tokens displayed in the dashboard as the adjudicator works, so reviewers can watch the agent reason in real time.
The Feature That Changed Everything: The Skeptic
Early versions of Liquet had a critical flaw we didn't notice until we started reading the verdicts carefully.
The adjudicator was too confident. Given both narratives and all evidence, it reliably produced a verdict — and then the human reviewer would read it and immediately see the one thing it missed: the seller's counter-claim about return shipping that, if true, changed the outcome entirely.
We added a Skeptic Pass: after the primary verdict, a second qwen3.7-max call receives only the losing party's narrative and the primary verdict. Its instruction is to generate the strongest possible case against the verdict. It has no access to the primary reasoning, only the evidence that favors the side that lost.
The first version still agreed with the primary verdict most of the time — because it could see the same evidence the adjudicator saw. The fix was forced adversarial asymmetry: give the skeptic only the losing narrative, nothing else. That produced real rebuttals. On cases where the skeptic raised a hard contradiction the primary pass had underweighted, the case correctly escalated.
A system with a built-in mechanism for arguing against itself is harder to fool than one that always confirms its own reasoning.
Stability Scoring: Running It Three Times on Purpose
Here's a non-obvious thing we discovered: a model that gives the same answer every time is not the same as a model that gives the right answer.
We ran the adjudicator on the same case file with temperature set to zero — perfectly deterministic — and got identical outputs across three runs. But when we introduced slight temperature variation and shuffled the order of evidence items in the system prompt between runs, something interesting happened: about 15% of cases produced different outcomes across the three passes.
Those cases were almost always the ones where a human reviewer, reading the brief, would say "I can see why someone could go either way here." The variance was real signal about genuine ambiguity — not noise.
So we kept it. Stability scoring runs three adjudication passes with slightly varied conditions, measures the fraction that agree, and applies a stability penalty to the effective confidence when they don't. A verdict that holds across all three runs earns its confidence score. One that splits is penalised. One that produces three different resolutions cannot pass the LIQUET gate regardless of what the nominal confidence number says.
effective_confidence = raw_confidence × stability_score
Ghost Cases: Grounding the Agent in History
One of the subtler features — and one of the most impactful — is ghost case injection.
Before the primary adjudication pass, Liquet queries the historical case database for disputes with matching category, overlapping order value bracket, and the same set of evidence types present. Up to five matching historical cases, with their final resolution and outcome, are injected into the adjudicator's system prompt as precedent examples.
The practical effect is that the model reasons over what the platform has actually decided before, not just its general prior about disputes. On categories where the platform has established a strong track record — "never arrived" cases with confirmed non-delivery scan carry a 92% buyer-favor rate in the database — the adjudicator grounds its reasoning in that pattern rather than starting from scratch.
When no close historical matches exist, the case adjudicates on first principles and Liquet says so explicitly in the brief.
Closing the Loop: Email-to-Resolution with No Human Touch
The full closed loop that we're most proud of:
- Buyer sends a complaint email
- The IMAP poller detects it, passes it to qwen3.7-max for parsing
- The parser extracts buyer narrative, order ID, dispute category, and any attached photos
- A new case is automatically submitted and queued for investigation
- The orchestrator runs the full pipeline
- On LIQUET: both buyer and seller receive a verdict email; a signed webhook fires to the external system
- On NON LIQUET: the human reviewer receives a decision brief and a HMAC-SHA256 signed one-click approval link
- The reviewer clicks approve or override — no separate login, no UI navigation required
- The case closes with a full immutable audit trail
End-to-end, zero human involvement on LIQUET cases. For the reviewer on NON LIQUET cases, the cognitive load is minimized: every piece of information needed to decide is pre-surfaced on one screen, with confidence breakdown, stability scores, and the exact reason the case abstained.
What Working with Qwen Cloud Taught Us
Model tier selection matters more than always reaching for the biggest model. The naive approach — route everything through qwen3.7-max — would have made the system 4–5× more expensive and no more accurate on the simple cases that make up the majority of volume. qwen3.6-flash at triage costs a fraction of a qwen3.7-max call and makes decisions that are good enough to route correctly. Using the right model for the right task is an architectural decision, not an afterthought.
Extended thinking is genuinely valuable, but only for the right calls. qwen3.7-max's extended chain-of-thought pass improved our adjudication quality measurably on complex cases with conflicting evidence. But for the skeptic pass — where we want a focused adversarial rebuttal, not an essay — it added latency without benefit. Knowing when to turn it on versus off took experimentation.
The QwenCloud API's reliability under concurrent load held up throughout. Running three stability passes concurrently, with a vision call happening in parallel, under our semaphore-limited concurrency of 3 simultaneous investigations, we didn't see unexpected rate limit failures once we respected the concurrency model. The structured output support for JSON schema enforcement made our Pydantic integration clean.
The Architecture: Why It's Built to Last
Liquet's core pattern — fast triage → deep evidence analysis → adversarial stress test → deterministic gate → human escalation with brief — is not specific to marketplace disputes. The same structure applies to insurance claim routing, vendor invoice disputes, content moderation appeals, and any domain where:
- Inputs are ambiguous and partially contradictory human narratives
- Some cases can be resolved confidently and some genuinely cannot
- Wrong confident answers are more dangerous than honest abstentions
- A human reviewer needs to be fast and correct on the cases that escalate
The domain models and policy document change. The orchestration, gate logic, and audit trail do not. We designed Liquet to be a reference implementation of a pattern, not a one-off tool.
What's Next
The gap between "useful demo" and "production system" in dispute resolution is: real EHR integration, multi-currency jurisdiction-aware policy selection, and longitudinal seller risk scoring. We know exactly where the seams are. The architecture is already async SQLAlchemy on top of SQLite — switching to PostgreSQL for real concurrent load is a DATABASE_URL change. The repository layer is already abstracted.
The next version of Liquet doesn't need a new system. It needs a real data feed, a real policy library, and time.
Try It
Live demo: http://liquet.43.98.167.71.sslip.io
Source code: https://github.com/nnam-droid12/Liquet
Deployed on Alibaba Cloud ECS, Singapore. QwenCloud API at dashscope-intl.aliyuncs.com/compatible-mode/v1.
Liquet knows when it's clear — and admits when it isn't.
Top comments (1)
Fascinating architecture. Using multiple specialized models instead of one general-purpose model is a strong design choice—it lets each model focus on a specific responsibility, making the overall system easier to evaluate, optimize, and maintain.
What stood out to me is the emphasis on structured reasoning and arbitration rather than just LLM responses. For AI systems making decisions that affect buyers and sellers, explainability, traceability, and consistent evaluation are just as important as raw model performance. This is a great example of designing AI around a workflow, not just a model.