DEV Community

Rustamjon Akhmedov
Rustamjon Akhmedov

Posted on

AIAuditFrauds: Building a Human-in-the-Loop AI Auditor on Qwen Cloud

#ai #qwen #agents #hackathon

For the Global AI Hackathon Series with Qwen Cloud (Track 4 · Autopilot Agent) I built AIAuditFrauds: a multi-agent AI that automates a corporate finance audit end-to-end — semantic search over 1,558 invoices, rule detectors, OFAC sanctions screening, a written report — while a human approves every consequential action. 100% on the Qwen stack: qwen-max + text-embedding-v4 (DashScope), AgentScope, DashVector, deployed on Alibaba Cloud ECS.

Try the Live Demo of AIAuditFrauds

The problem: AI you can trust with money

Corporate finance teams drown in invoices, and fraud hides in the noise: the same invoice paid twice, a "vendor" that doesn't exist, an amount that quietly breaks policy, a wire pushed through at 3 a.m., a payee on a sanctions list. No human can eyeball tens of thousands of rows.

But finance is exactly where you cannot let an AI act on its own. An LLM that hallucinates a paragraph is embarrassing; an LLM that flags the wrong vendor as fraudulent — or misses a sanctioned payee — is a lawsuit. So the design goal became the whole product: automate the workflow end-to-end, but put a human checkpoint in front of every consequential action.

And one architectural rule I never broke:

The deterministic detectors decide the audit outcome. Qwen plans, narrates and answers questions. If the model is ever down, the audit still runs.

What it does — one mission, two human gates

You type an ambiguous, free-text mission: "find duplicate vendor payments and suspicious spending this quarter."

AIAuditFrauds mission console where the auditor types a free-text audit mission

Gate 1 — the plan. Qwen (qwen-max via DashScope) interprets the mission and drafts a concrete audit plan. Nothing runs until the auditor approves, edits, or rejects it.

Gate 1 showing the Qwen-generated audit plan awaiting human approval

Evidence. A team of AgentScope specialists takes over, each step attributed live in the timeline:

  • Transaction Screening → semantic search in DashVector over Qwen text-embedding-v4 vectors
  • Spend Analysis → SQL aggregation in PostgreSQL
  • Risk Triage → deterministic detectors (exact/near duplicates, policy limits, ghost vendors, off-hours payments) + live screening against the US Treasury OFAC sanctions list

Agent timeline showing each specialist agent's step during the audit run

The run above flagged 595 suspicious invoices worth $33M — each with reasons and evidence you can drill into:

Invoice evidence review panel listing flagged invoices with reasons

Gate 2 — the human decides. The auditor approves or rejects items one by one. Only approved invoices are written — a single idempotent, audited transaction. The AI never touches the ledger.

Gate 2 human decision screen where the auditor approves or rejects flagged invoices

Report. Qwen writes the closing narrative, and an Audit Assistant answers follow-up questions grounded in the actual evidence ("Which flagged invoice is the most urgent to verify, and why?").

Closing report with the Audit Assistant answering a follow-up question

The architecture

System architecture diagram of AIAuditFrauds on the Qwen Cloud stack

Everything hinges on one seam: a typed Server-Sent-Events contract — plan → gate 1 → tools → proposal → gate 2 → gated write → report → done. The frontend, store and API depend only on that event sequence, so the model, agent and data layers could evolve freely underneath while 124 contract tests kept everything honest (the whole suite runs with zero cloud credentials).

Working with the Qwen Cloud stack — field notes

This was my first project going all-in on the Qwen ecosystem. Honest notes from the journey:

DashScope (Qwen API). One key from the Qwen Cloud console drives both chat and embeddings. text-embedding-v4 returns clean 1024-dim vectors; embedding 1,558 invoices took under two minutes in batches of 10.

How to fix the 401 invalid_api_key error

The one gotcha for international accounts: use the intl endpoints, or you'll stare at 401 invalid_api_key wondering what you did wrong:

# native SDK
dashscope.base_http_api_url = "https://dashscope-intl.aliyuncs.com/api/v1"

# OpenAI-compatible mode (what AgentScope uses under the hood)
base_url = "https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
Enter fullscreen mode Exit fullscreen mode

AgentScope 2.x. Alibaba's multi-agent framework had a major rewrite, so tutorials from 1.x will mislead you — read the installed source. The API I landed on:

AgentScope 2.x agent setup 
from agentscope.agent import Agent  # Agent IS the ReAct agent
from agentscope.model import DashScopeChatModel
from agentscope.credential import DashScopeCredential
from agentscope.tool import Toolkit, FunctionTool, ToolResponse

model = DashScopeChatModel(
    credential=DashScopeCredential(api_key=KEY, base_url=INTL_COMPAT_URL),
    model="qwen-max",
)
agent = Agent(name="RiskTriageAgent", system_prompt=..., model=model,
              toolkit=Toolkit(tools=[FunctionTool(screen_vendor_sanctions)]))
Enter fullscreen mode Exit fullscreen mode

Three things that cost me debugging time: Msg.content must be a list of content blocks (not a string), tool functions must return a ToolResponse (not a dict), and the credential's base_url decides which region you hit. Once wired, an agent calling a real OFAC-screening tool worked on the first live run — and AgentScope's built-in external tool-confirmation flow maps beautifully onto human-in-the-loop designs.

DashVector. The pleasant surprise of the project. Free trial cluster, created a 1024-dim cosine collection in the console, and the SDK is tiny:

DashVector upsert and query 
coll = dashvector.Client(api_key=KEY, endpoint=ENDPOINT).get("transactions")
coll.upsert([dashvector.Doc(id=inv.invoice_id, vector=vec, fields={...})])
hits = coll.query(query_vector, topk=8, output_fields=["vendor_name", "amount"])
Enter fullscreen mode Exit fullscreen mode

Being built by the same team as Qwen, it pairs naturally with Qwen embeddings — no dimension gymnastics, no separate index management.

Alibaba Cloud ECS. The whole system deploys as one Docker Compose stack (app + PostgreSQL) on a $0.02/hour ECS instance in Singapore, behind Caddy for automatic Let's Encrypt HTTPS. Total cloud bill for the entire hackathon: under $10. A fun proof-of-deployment trick: Alibaba's internal metadata service (http://100.100.100.200) is only reachable from inside Alibaba Cloud — one curl attests your instance identity.

What I'd tell you if you're building on Qwen Cloud

  1. Pin your AgentScope version and introspect the installed API — the 2.x rewrite is good, but docs lag. inspect.signature() is your friend.
  2. Design so the LLM is a planner/narrator, not the source of truth. Deterministic logic decides; Qwen explains. Your demo will survive any quota hiccup.
  3. Keep one region. DashScope intl + DashVector + ECS + OSS all in Singapore = no cross-region surprises and free internal traffic to OSS.
  4. Make your test suite credential-free. An in-memory repository behind the same interface as Postgres + DashVector meant CI never needed a single key.
  5. Human gates are a feature, not friction. The two-gate design is the reason a finance team could actually deploy this.

The result

A production-posture autopilot for finance audits: live on Alibaba Cloud, HTTPS domain, 124 green tests, graceful fallbacks, idempotent writes — and a human in command of every dollar-touching decision.

GitHub logo proga100 / aiauditfrauds

Multi-agent finance fraud auditor on Qwen + AgentScope + DashVector, deployed on Alibaba Cloud (Qwen Cloud Hackathon, Agent Society)

AIAuditFrauds

A multi-agent AI auditor that automates the corporate finance-audit workflow end-to-end — hunting duplicate payments, ghost vendors, policy breaches, off-hours transactions and OFAC sanctions risk — with human-in-the-loop checkpoints at every critical decision.

Built for the Global AI Hackathon Series with Qwen Cloud · Track: Autopilot Agent · Runs 100% on the Qwen / Alibaba Cloud stack.

The idea — and why it matters

Corporate finance teams drown in invoices, and fraud hides in the noise: the same invoice paid twice, a payment to a vendor that doesn't really exist, an amount that quietly breaks policy, a wire pushed through at 3 a.m., a payee on a sanctions list. A human can't eyeball…

View on GitHub

Built for the Global AI Hackathon Series with Qwen Cloud — Track 4: Autopilot Agent. The project concept existed before the hackathon; during the submission period it was rebuilt end-to-end on the Qwen stack (DashScope, AgentScope, DashVector, ECS). Development was AI-assisted with Claude Code.

Top comments (0)