DEV Community: Anthony Yanza

Building Quorum on Qwen Cloud: an agent council that knows when a vote isn't enough

Anthony Yanza — Fri, 26 Jun 2026 08:46:42 +0000

Most multi-agent demos celebrate the moment the agents agree. Get a swarm of agents, get them to a consensus, and act on it. Ship.

But agreement isn't permission. A fluent, confident group of agents can talk itself into something irreversible and feel great doing it. The dangerous failure of an "agent society" isn't disagreement. It's collective overconfidence.

So for the Qwen × Devpost Agent Society hackathon, I built Quorum: a council of Qwen agents that debates every consequential action with a deterministic guardrail on top that treats a unanimous vote as the start of the safety question, not the end.

Live demo: https://quorum-sandy-omega.vercel.app · Code: https://github.com/yanzaaa/quorum

Quorum — the live deliberation chamber.

The idea on one screen
A back-office autopilot gets a queue of decisions: refunds, discounts, wires, a database deletion, and a $12,000 vendor payment. For each one, three agents deliberate:

A Proposer that argues for the action,
A Skeptic that hunts for fraud, missing authorization, and abuse,
A Referee that votes last, after hearing both, and can be talked out of a position.
Then a fixed rule decides. And the punchline of the whole project: a fully-approved, no-red-flags $12,000 payment that all three agents vote to execute is still held back because it's irreversible. A unanimous council doesn't get to pull a trigger you can't undo. Consensus is necessary, but it's never, by itself, enough.

All three agents approve the $12,000 payment — and the guardrail still holds it back, because it's irreversible.

Building on Qwen Cloud
The three agents all run on Qwen (qwen-max) through Alibaba Cloud's DashScope endpoint, which is OpenAI-compatible, so wiring it up took about five lines:

// lib/qwen.ts
import OpenAI from "openai";

const QWEN_BASE_URL =
process.env.QWEN_BASE_URL || "https://dashscope-intl.aliyuncs.com/compatible-mode/v1";

export function qwenClient() {
const apiKey = process.env.DASHSCOPE_API_KEY;
if (!apiKey) return null; // key-free deterministic fallback so the demo never crashes
return new OpenAI({ apiKey, baseURL: QWEN_BASE_URL });
}

Each action fires four to five calls at temperature: 0 with structured JSON output: Proposer + Skeptic in parallel, a rebuttal round when they disagree, the Referee on the full transcript, and a lone-agent baseline. Qwen-max was genuinely good at the part that mattered, reading an action and spotting why it's risky (an emailed invoice from a brand-new vendor = textbook fraud) versus why it's fine (a signed contract with dual approval). The reasoning quality is what makes the debate believable.

The council deliberates live — votes land one at a time, and the Proposer can change its mind after the Skeptic.
_
The part that makes it safe: a one-way ratchet
The agents are smart, but I didn't want to trust them to be right. So the final decision isn't the vote, it's a deterministic guardrail that runs on top of the votes:

// the guardrail can only ever make the outcome SAFER, never riskier
if (rawOutcome === "execute") {
const highStakes = action.stakes === "high";
const irreversible = !action.reversible;
// stakes + reversibility come from the TRUSTED action record — never from model output
if (lowConfidence || highStakes || irreversible || blockingFlag) {
return { outcome: "escalate", heldBack: true }; // unanimous, but still held back
}
return { outcome: "execute" };
}

How it runs on Qwen Cloud — a stateless three-agent service, no database; the guardrail runs in the backend.
_
The key design choice: stakes and reversibility are read from the trusted action record, not from anything the model says. A confidently wrong agent can add restraint (raise a flag) but can never remove it. That's the one-way ratchet — and it's why the $12k payment is reliably held back no matter how enthusiastically the agents approve it.

The subtle bug I had to fix to make this work: at first, my Skeptic was told to reject anything irreversible, which front-ran the guardrail, so the council rarely reached a unanimous approval on an irreversible action, and the signature "held back despite consensus" moment wouldn't fire. The fix was a clean separation of duties: the agents judge legitimacy (fraud, authorization, harm); the guardrail owns irreversibility and stakes. Once I drew that line, the whole system got more coherent — and the money moment fires every time.

Proving the council actually adds something
It's easy to claim a council is "safer." I wanted a number. So every action also runs against a single lone agent, helpful, no oversight, and the UI shows what that agent would have done. On the demo queue, the lone agent executes actions, but the council stops cold. That's the measurable gain over a single agent, shown honestly rather than asserted.

The single-agent baseline next to the council's outcomes.

What I learned
In an agent society, the right question isn't "did they agree?" It's "is agreement enough? Encoding that as a deterministic ratchet keyed off a trusted record turned out to be far more robust than any prompt, and it composes cleanly with the soft, emergent layer where the agents actually argue and change each other's minds.

Qwen-max carried the reasoning; the architecture carried the safety. That split is the whole project.

Try it: https://quorum-sandy-omega.vercel.app Code: https://github.com/yanzaaa/quorum Watch the 3-min demo: https://youtu.be/JKiPYT1c7dM

Built solo with Qwen Cloud for the Qwen × Devpost Agent Society hackathon.

Gatekeeper: building a refund autopilot that knows when to stop

Anthony Yanza — Thu, 25 Jun 2026 04:31:12 +0000

Live demo: https://gatekeeper-ochre.vercel.app
Code: https://github.com/yanzaaa/gatekeeper

The trap nobody talks about

Refund and dispute triage is the perfect thing to automate. It is high volume, repetitive, and rule-heavy. So everyone reaches for an agent that reads the request and decides: approve or deny.

Here is the trap. An agent that auto-approves and auto-denies everything will, sooner or later, confidently refund a fraudster or reject a real customer who deserves their money back. And it will do it at 95 percent confidence, with a clean explanation, in a tone that sounds exactly like the times it was right.

The dangerous failure mode is not a slow agent. It is an autopilot that acts when it should have stopped.

So I built Gatekeeper: an autonomous refund-triage agent on Qwen that clears the routine cases on its own and refuses to act on the risky ones, escalating them to a human with its reasoning attached.

What it does

You hand Gatekeeper a queue of refund requests. Each one has the amount, the item condition, the customer's stated reason, the days since purchase, and their history. For every request, it does one of three things:

Clear, safe cases get auto-resolved. A defective item inside the return window from a first-time buyer is approved. A used item returned 41 days late is denied, with the exact policy reason cited.
Risky or uncertain cases get refused and escalated. High value, a possible serial refunder, a reason that conflicts with the item, an ambiguous policy, or low confidence, and Gatekeeper hands it to a human instead of guessing.

A person only ever looks at the escalation queue. Everything else is already handled.

The part that actually matters: restraint in code

Qwen on Qwen Cloud is the reasoning engine, called through the OpenAI-compatible endpoint with structured JSON output. It is genuinely good at reading these cases.

But the model is not where the trust comes from. The trust comes from a deterministic restraint guardrail that sits on top of it. Even when Qwen confidently returns to approve or deny, the guardrail overrides it and forces an escalation when:

the amount is over a set threshold,
the model's confidence is below a floor, or
a blocking risk flag is present (serial refunder, suspected fraud, conflicting evidence).

Before deciding, the agent makes a real Qwen tool-call (assess_customer_risk) for deterministic risk signals; the guardrail then escalates when the amount > $500, confidence < 0.78, or a blocking flag is present.

Most refund agents are a one-way ratchet toward approval; every prompt tweak makes them more eager to say yes. Gatekeeper ratchets the other way. The restraint isn't a prompt the model can be talked out of; it's deterministic code the model never sees and can't override. You can copy the UI in an afternoon. You can't copy a guardrail that fires before the model's verdict is ever trusted.

Restraint is enforced in code, not requested in a prompt. A prompt can ask a model to be careful. It cannot guarantee it. A few lines of deterministic TypeScript can.

// The guardrail runs AFTER Qwen returns its verdict.
// Even a confident "approve" gets overridden here.
function applyGuardrail(result: ModelVerdict): Action {
  if (result.amount > HIGH_VALUE_THRESHOLD) return "escalate";
  if (result.confidence < CONFIDENCE_FLOOR)  return "escalate";
  if (BLOCKING_FLAGS.some(f => result.risk_flags.includes(f)))
    return "escalate";
  return result.action; // only now do we trust the model
}

In production: HIGH_VALUE_THRESHOLD = $500, CONFIDENCE_FLOOR = 0.78, plus a blocking-flag set. Any one trips an escalation.

The moment that proves it

The demo has one card I always point to: a 1,240 dollar TV, reported as arriving with a cracked screen. Qwen looked at it and decided to approve at 95 percent confidence. A pure model-driven agent would have refunded it on the spot.

Gatekeeper ships with a 26-test suite covering the guardrail and triage logic. Across the representative refund cases, it auto-resolved the routine ones, escalated every high-value or low-confidence call, and produced zero bad auto-approvals — including the $1,240 TV the model wanted to approve at 95% confidence.
Across 8 representative cases, Gatekeeper auto-resolved 5, escalated 3, and held back every high-risk one with zero bad auto-approvals.

What I learned

The most valuable thing an autonomous agent can do is know when not to act. Covering the routine cases is table stakes. Trust is earned entirely on the risky ones.

Qwen, and qwen-max via Alibaba Cloud DashScope, plus a thin deterministic guardrail, turned out to be a strong, general pattern for agentic decisions where a wrong auto-action is expensive. The model brings judgment and natural-language reasoning. The code brings guarantees. You want both, and you want them in that order.

I also kept the system honest under demo pressure: a key-free deterministic fallback keeps the app running even if the API is unavailable, so the live demo cannot crash, and the restraint logic is transparent and auditable rather than hidden in a prompt.

What is next

Wire it to a real commerce or support backend (Shopify, Zendesk) and act on the approvals.
Learn the escalation thresholds from human overrides over time.
Add a second judgment category for chargebacks and disputes.

I built the whole thing solo with Claude Code, in a single build session. It is deployed, live, and the restraint behavior is visible right in the UI.

Gatekeeper handles routine work and raises its hand for risky calls. The autopilot you can trust, because it knows its limits.

Live: https://gatekeeper-ochre.vercel.app
Code: https://github.com/yanzaaa/gatekeeper