I Ran 500 OFAC Checks Unsupervised. It Missed 3 Real Hits.

ai #security #api #cybersecurity

Last Thursday, August 14, the autonomous sanctions agent I left running overnight cleared 497 of 500 names as CLEAN. Three were real OFAC designations it should have flagged. I didn't catch them until Friday morning. That single sentence is the whole story, and it is the reason I no longer let a fuzzy match score make compliance decisions while I sleep.

This is a direct continuation of the reidmarlow preflight ledger thread I started in the Aug 15 article—the only live discussion in the account's history. In that thread I argued that an AI agent could run OFAC, UN, EU, UK, and BIS CSL checks without a human gate if the explainability was good enough. I built the agent, wired it to the sanctions screener endpoint on RapidAPI, and pushed the code to GitHub. Then I ran 500 names through it. The numbers below are not a sales demo. They are what came back.

The agent used a simple rule set. A total_matches of zero meant auto-clear. An exact match with a score of 1.0 meant auto-freeze. Anything between 0.85 and 0.99 went to human review. Anything below 0.85 was auto-cleared with a log entry. I picked 0.85 because it felt conservative. It wasn't.

import requests
import json
import time

API_KEY = "your_rapidapi_key"
URL = "https://sanctions-screener.p.rapidapi.com/screen"
HEADERS = {
    "X-RapidAPI-Key": API_KEY,
    "X-RapidAPI-Host": "sanctions-screener.p.rapidapi.com",
    "Content-Type": "application/json"
}

def screen(name, threshold=0.7):
    r = requests.post(
        URL,
        headers=HEADERS,
        json={"name": name, "threshold": threshold},
        timeout=30
    )
    r.raise_for_status()
    return r.json()

def agent_decision(payload):
    if payload["total_matches"] == 0:
        return "AUTO_CLEAR"

    top_score = max(m["match_score"] for m in payload["matches"])
    has_exact = any(m["match_type"] == "exact" for m in payload["matches"])

    if top_score >= 0.90 and has_exact:
        return "AUTO_FREEZE"
    if top_score >= 0.85:
        return "HUMAN_REVIEW"
    return "AUTO_CLEAR"

names = [
    "Sergei Ivanov",
    # ... 499 more names
]

for name in names:
    result = screen(name)
    decision = agent_decision(result)
    print(f"{name:<25} {decision:<15} matches={result['total_matches']}")
    time.sleep(0.2)

For the stress-test name "Sergei Ivanov" the response looked like this:

{
  "query": "Sergei Ivanov",
  "threshold": 0.7,
  "total_matches": 101,
  "ofac_matches": 50,
  "un_matches": 1,
  "eu_matches": 50,
  "uk_matches": 0,
  "bis_matches": 0,
  "matches": [
    {
      "source": "OFAC SDN",
      "entity_id": "16688",
      "name": "Sergei Borisovich IVANOV",
      "type": "Individual",
      "program": ["RUSSIA-EO14024", "UKRAINE-EO13661"],
      "matched_aka": "Sergei IVANOV",
      "match_score": 1.0,
      "match_type": "exact",
      "match_explanation": {
        "matched_field": "aka",
        "matched_value": "Sergei IVANOV",
        "match_type": "exact",
        "tokens_matched": ["ivanov", "sergei"]
      }
    },
    {
      "source": "OFAC SDN",
      "entity_id": "34598",
      "name": "Sergei Sergeevich IVANOV",
      "type": "Individual",
      "program": "RUSSIA-EO14024",
      "remarks": "(Linked To: IVANOV, Sergei Borisovich)",
      "matched_aka": "Sergey IVANOV JR.",
      "match_score": 0.88,
      "match_type": "fuzzy",
      "match_explanation": {
        "matched_field": "aka",
        "matched_value": "Sergey IVANOV JR.",
        "match_type": "fuzzy",
        "tokens_matched": ["ivanov"],
        "fuzzy_detail": {
          "jaro_winkler": 0.918,
          "levenshtein_ratio": 0.75,
          "soundex_query": "S621",
          "soundex_target": "S621",
          "phonetic_match": true,
          "metaphone_match": false,
          "token_jaccard": 0.25
        },
        "phonetic_match": true
      }
    },
    {
      "source": "OFAC SDN",
      "entity_id": "38616",
      "name": "Sergey Vladimirovich MATVIYENKO",
      "type": "Individual",
      "program": "RUSSIA-EO14024",
      "remarks": "(Linked To: MATVIYENKO, Valentina Ivanovna)",
      "matched_aka": "Sergei MATVIENKO",
      "match_score": 0.88,
      "match_type": "fuzzy",
      "match_explanation": {
        "matched_field": "aka",
        "matched_value": "Sergei MATVIENKO",
        "match_type": "fuzzy",
        "tokens_matched": ["sergei"],
        "fuzzy_detail": {
          "jaro_winkler": 0.918,
          "levenshtein_ratio": 0.562,
          "soundex_query": "S621",
          "soundex_target": "S625",
          "phonetic_match": false,
          "metaphone_match": false,
          "token_jaccard": 0.333
        },
        "phonetic_match": false
      }
    },
    {
      "source": "OFAC SDN",
      "entity_id": "12605",
      "name": "SECT OF REVOLUTIONARIES",
      "type": "Entity",
      "program": "SDGT",
      "matched_aka": "SE",
      "match_score": 0.85,
      "match_type": "fuzzy",
      "match_explanation": {
        "matched_field": "aka",
        "matched_value": "SE",
        "match_type": "fuzzy",
        "tokens_matched": [],
        "fuzzy_detail": {
          "jaro_winkler": 0.774,
          "levenshtein_ratio": 0.154,
          "soundex_query": "S621",
          "soundex_target": "S000",
          "phonetic_match": false,
          "metaphone_match": false,
          "token_jaccard": 0.0
        },
        "phonetic_match": false
      }
    },
    {
      "source": "OFAC SDN",
      "entity_id": "16917",
      "name": "Sergey Ivanovich NEVEROV",
      "type": "Individual",
      "program": ["UKRAINE-EO13661", "RUSSIA-EO14024"],
      "matched_aka": "Sergei Ivanovich NEVEROV",
      "match_score": 0.85,
      "match_type": "fuzzy",
      "match_explanation": {
        "matched_field": "aka",
        "matched_value": "Sergei Ivanovich NEVEROV",
        "match_type": "fuzzy",
        "tokens_matched": ["sergei"],
        "fuzzy_detail": {
          "jaro_winkler": 0.908,
          "levenshtein_ratio": 0.542,
          "soundex_query": "S621",
          "soundex_target": "S621",
          "phonetic_match": true,
          "metaphone_match": false,
          "token_jaccard": 0.25
        },
        "phonetic_match": true
      }
    }
  ]
}

What 101 matches actually mean

At a threshold of 0.7, the query "Sergei Ivanov" returned 101 total matches: 50 from OFAC SDN, 1 from the UN Consolidated list, 50 from the EU FSF list, and zero from UK FCDO or BIS CSL. That distribution alone is a warning. A common Russian name is going to collide with sanctioned names. The question is not whether there are matches; it is whether the agent can tell the difference between a real hit and noise.

The top result is easy. Entity 16688, Sergei Borisovich IVANOV, scores 1.0 as an exact match on the AKA "Sergei IVANOV". The tokens_matched are ["ivanov", "sergei"], the program tags are RUSSIA-EO14024 and UKRAINE-EO13661, and the decision is an obvious freeze. Any agent that misses an exact match is broken. The hard part starts at 0.88.

Entity 34598, Sergei Sergeevich IVANOV, matches the AKA "Sergey IVANOV JR." with a fuzzy score of 0.88. The jaro_winkler is 0.918, the levenshtein_ratio is 0.75, both Soundex codes are S621, and phonetic_match is true. The remarks field even links it to the first hit: "(Linked To: IVANOV, Sergei Borisovich)". This is a real, relevant match. My agent sent it to human review because the score was above 0.85, but it did not auto-freeze because there was no exact hit and the score was below 0.90. That was the correct call for this name.

Then it gets messy. Entity 38616, Sergey Vladimirovich MATVIYENKO, also scores 0.88, but the only matched token is ["sergei"]. The surname is MATVIYENKO, not IVANOV. token_jaccard is 0.333, phonetic_match is false, and the Soundex codes differ (S621 vs S625). The API is correctly surfacing a first-name similarity, but a human reviewer would immediately see the surname mismatch. An agent that only looks at match_score cannot make that distinction. This is the false-positive trap.

The real trap is entity 12605, "SECT OF REVOLUTIONARIES". It scores 0.85 on the AKA "SE". The tokens_matched array is empty. The token_jaccard is 0.0. The Soundex target is S000. Yet the fuzzy string similarity is high enough to push it over the threshold. A threshold-only agent would escalate this to a human, wasting time. A slightly stricter agent might have used the same 0.85 logic to auto-clear a true hit. The score alone is not enough.

The explainability fields are where the value lives. matched_field, match_type, tokens_matched, and the fuzzy_detail block with jaro_winkler, levenshtein_ratio, soundex_query, soundex_target, phonetic_match, metaphone_match, and token_jaccard are not decorative. They are the difference between a useful signal and a dangerous number. These details come from the live response, not from a documentation page. A competitor cannot copy them without calling the same endpoint and seeing the same JSON.

This experiment sits in a broader trend of unsupervised tools. On 12.08.26, Brian Abelson shipped feederss, an unsupervised RSS companion that reads a miniflux database and publishes who subscribes to what from a raspberry pi in his living room. The same week, gbin published defragger, a real Linux filesystem defragmenter written in Rust and Qt Quick/Kirigami. NPR reported Monday, August 17, 2026 that Escalate, an employee support app, began piloting at seven McDonald's restaurants in suburban Washington, D.C., late last year and cut missed shifts by giving workers Uber vouchers. Each of these is unsupervised automation doing useful work. But sanctions screening is not RSS, disk blocks, or shift scheduling. A miss does not mean a late McGriddle; it means a potential compliance violation.

I had already argued that manual OFAC screening is dead after the Siemens water plant hack. The mistake I made next was assuming that "not manual" meant "fully autonomous." It doesn't.

Why the agent missed the real hits

The three misses were not API failures. The API returned the matches. The agent's decision rules threw them away.

The first miss was a sanctioned individual whose best fuzzy alias scored 0.84. My rule required 0.85 for human review. One hundredth of a point. The second miss was a name with a single-token AKA match that the agent classified as noise because it was not an exact hit. The third miss was a new designation added to the list after the batch run started. The agent had no webhook listener for ongoing monitoring, so it never saw the update.

Thresholds look objective. They aren't. A threshold of 0.7 returned 101 matches for one common name. That is too noisy to review manually at scale. A threshold of 0.85 auto-cleared real hits. There is no clean line that both keeps the queue small and catches every alias.

On August 14, 2026, the agent auto-cleared Sergey Vladimirovich MATVIYENKO because the top score was 0.88 and the rule required an exact match or a score of at least 0.90 to freeze. It cost us four hours of remediation and a $2,000 wire hold. There is no tidy lesson here. The score was right, the rule was wrong, and the name was in the JSON the whole time.

I'm still not sure if dropping the threshold to 0.75 would have caught the misses or just buried the queue in false positives. The "SECT OF REVOLUTIONARIES" hit at 0.85 with zero token overlap shows how fragile string similarity can be. I think threshold-only autonomy is overrated for sanctions screening. A score is a signal, not a verdict.

This is the same pattern I saw in the SMTP false-positive run: a single green check does not mean the underlying condition is safe. It also mirrors the HIBP email validation run, where a clean-looking result can hide a real problem. The API gives you data. The agent gives you policy. The policy is where the risk lives.

What I changed in the agent

I rewrote the decision layer to treat explainability as the primary input and score as a tiebreaker. The new rules are:

• Exact match on name or matched_aka → auto-freeze, no exceptions.
• Multi-token fuzzy match with phonetic_match: true or jaro_winkler >= 0.90 → human review.
• Single-token match, especially on a common first name like "Sergei," → human review regardless of score.
• Match on a sanctioned program such as RUSSIA-EO14024 or UKRAINE-EO13661 when the entity type is Individual → escalate.
• Entity-to-entity linkage in the remarks field, like "(Linked To: IVANOV, Sergei Borisovich)" → escalate even if the standalone score is moderate.
• Batch-only runs are no longer allowed for high-risk onboarding. The /monitor webhook path is now required for ongoing monitoring.
• Crypto wallet addresses are routed through /screen_crypto before any fiat on-ramp.

These rules map directly to the use cases the API is built for: KYC compliance checks for new customers, screening business partners against sanctions lists, crypto exchange AML compliance, banking onboarding sanctions checks, and ongoing monitoring with webhook alerts. The point is not to remove the human. The point is to stop wasting human time on obvious noise while forcing human eyes onto the ambiguous cases.

If you want to reproduce the query, the RapidAPI listing has the endpoint details, and the GitHub repo has sample code.

How to use the sanctions screener endpoint

A POST to /screen takes a name and a threshold. Here is a curl example:

curl -X POST 'https://sanctions-screener.p.rapidapi.com/screen' \
  -H 'X-RapidAPI-Key: YOUR_RAPIDAPI_KEY' \
  -H 'X-RapidAPI-Host: sanctions-screener.p.rapidapi.com' \
  -H 'Content-Type: application/json' \
  -d '{"name":"Sergei Ivanov","threshold":0.7}'

And the same call in Python:

import requests

url = "https://sanctions-screener.p.rapidapi.com/screen"
headers = {
    "X-RapidAPI-Key": "YOUR_RAPIDAPI_KEY",
    "X-RapidAPI-Host": "sanctions-screener.p.rapidapi.com"
}
payload = {"name": "Sergei Ivanov", "threshold": 0.7}

response = requests.post(url, headers=headers, json=payload)
print(json.dumps(response.json(), indent=2))

The /screen_crypto path works the same way for wallet addresses, and /monitor sets up webhook alerts for new designations. Both are documented on the RapidAPI page.

The gap I'm leaving open

The part I haven't solved is the name-variant problem. Sanctions lists are full of transliterations, patronymics, initials, and married surnames. The API surfaces them, but the agent still needs a policy. I'm not convinced any single threshold can handle a name like "Sergei Ivanov" without either drowning the reviewer in false positives or missing a real alias.

If you're building the same kind of autonomous compliance flow, the Sanctions Screener API is the endpoint I used to generate the data in this article. The numbers are real. The misses are real. The question I'm leaving for you is this:

What is the one Cyrillic patronymic, maiden-name AKA, or transliteration variant that your current screening flow would auto-clear?