A risk score is a reason to look: building a fraud investigator on a graph

Ayush Kumar — Fri, 25 Sep 2026 06:19:27 +0000

HHGOA 2026 · TigerGraph Partner Challenge. Repo: github.com/Cometbuster4969/hacker-house ·
Demo:

The task handed us 20 alerts and a warning buried in the README: most transactions with a risk
score above 0.7 are legitimate. Our first agent took the obvious route — gather graph evidence,
ask an LLM to reason, let rules fill the gaps. It called 17 of 20 alerts fraud. Half were
legitimate. So we threw the verdict layer away and started from one question:
what does the bank actually know?

What we built

A deterministic investigation agent. For each alert it forms hypotheses, calls 7 tools (episode
reconstruction, device-ring fan-out, closed-case support, exposure, policy, memory), and emits one
JSON answer file: investigation record, evidence, findings, decision, actions, and the next best
action before and after more evidence arrives. policy.decide() is a pure function over the case
state — it is the only thing that writes actions or flags a SAR.

20/20 answers: 8 fraud, 9 legitimate, 3 left uncertain, 3 SARs, 181 tool calls, 0 LLM tokens in the decision path.
58 mechanical contract checks, 20 files / 0 violations via python main.py validate.
87 passed / 2 skipped unit+integration tests on the minimal install, no dataset or key needed.

Architecture

alert -> hypothesis loop -> tools (episode | device ring | closed-case support | exposure)
-> policy.decide() # pure function, R1-R10 + routing thresholds
-> answer JSON -> TigerGraph loader -> FastAPI dashboard
-> case memory (retrieved as priors on the next run)

text

The LLM is optional and only ever narrates; verdicts and actions come from the engine, so an
explanation can never contradict an action.

How we used TigerGraph

Schema in tigergraph/schema.gsql: 8 vertex types — Customer, Card, Transaction,
DeviceProfile, EmailDomain, BillingRegion, ClosedCase, InvestigationCase — with the
evidence traversals in tigergraph/queries/engine_queries.gsql (2-hop device fan-out via
FROM_DEVICE, burst aggregation, closed-case support). scripts/deploy_schema.py installs to
Savanna; scripts/load_to_tigergraph.py writes all 20 cases as InvestigationCase vertices
(written_to_graph: 20, snapshot in evidence/tg_live_check.json). The same graph doubles as the
agent's memory: the next investigation starts from vertices, not a cold re-read of CSVs.

What the graph found that no label did

Sub-$500 structuring. Four online purchases in 30 minutes, each a different amount between $456 and $499. Organic bursts repeat one price ($499.95 × 4); these don't. 22 such bursts exist in the corpus, 5 are confirmed closed cases — HHG-006 is another. A population sweep found the same shape on 3 other customers' cards the previous month, so the agent files it as undocumented under R9.
An anonymous-proxy device ring. One Samsung SM-G935F profile, marked New on every account, behind an anonymous proxy, on 20 cards of 20 customers in 30 days. HHG-014 sits on it: the agent puts the other 19 cards under monitoring and escalates.

We were as careful about false rings: "Windows / Chrome / 1920×1080" is shared by thousands of
honest people, so cards link only through hardware-specific devices or anomalous shared use.

Results, measured honestly

We replayed 544 October closed cases (400 confirmed, 144 cleared) through the whole agent,
models trained only on Jul–Aug, the calibrator on Sep, and every alert presented as a plain
risk-score trigger on a random transaction of the episode — harder than a real alert.

Measure	Result
Transaction model, Sep–Oct holdout (193,323 txns, 3.65% fraud)	ROC-AUC 0.947 vs 0.861 for the bank's risk score; Brier 0.018
Verdict accuracy on decided cases	0.82 (0.73 if every uncertain counts as wrong), 10.5% uncertain
Episode reconstruction vs analyst lists	mean Jaccard 0.82, exact 63%
Pattern accuracy on fraud verdicts	0.77
Blocking on cases the bank cleared	8.3% (vs 64.5% of confirmed cases blocked)
SAR agreement with analysts	0.91
Cost	8.1 tool calls and 0.17 s per case; 95.6 s for all 544

Ground truth is the bank's own 5,565 closed cases (4,665 confirmed fraud) over 590,742
transactions and 144,432 identities — the only confirmed outcomes in the data. Features are
strictly causal: a closed case counts only once it was closed at alert time. The dataset's
"customer" is an issuer-level bucket pooling many real people, so identity uses the card +
billing-region + account-open-day key.

Then the same engine swept Nov–Dec with no case pack and opened 15 alerts of its own: 12
structuring bursts, the full SM-G935F ring (28 customers, 60 transactions), one thin candidate ring
left for a human, one card-testing sequence.

What we learned

A score is a reason to look, not a verdict. Our first run's 17/20 fraud rate was the model being rational about a misleading feature.
Separating policy from reasoning is the whole product. Once rules are code, the explanation can't drift from the action, and a validator can grade all 20 files mechanically.
Uncertainty has to be an output. 3 cases stayed open; forcing them to a label costs precision you can't see in aggregate accuracy.
Build for re-derivation. Every number above is regenerated from checked-in artifacts; CI runs the suite on a clean clone with no graph and no key.
The graph earns its place on multi-hop questions — device fan-out and shared-merchant bursts — not on the tabular ones.

Deploy the case vertices for the whole corpus and let recall improve via TigerVector on
InvestigationCase, lift fraud recall (0.65) with a sequential model, and run the agent on live
alerts behind the dashboard.

Data note: a public column extract of the official HHGOA_IEEE files; never the public IEEE-CIS
labels. MIT-licensed.

DEV Community: Ayush Kumar