DEV Community

Sattyam Jain
Sattyam Jain

Posted on

Build a per-locale red-team harness for your LLM agent (before you trust the English number)

#ai #llm #security #showdev

A 2026 Appen study (arXiv:2605.23157) showed that LLM safety rankings do not survive a language switch — flip the prompt language and the "most vulnerable" model changes places, across 52,272 human ratings, with no rescaling able to recover the English order. A separate Hinglish red-team (arXiv:2505.14226) hit ~99% text attack-success using code-mixed phonetic perturbations.

The practical consequence: an English-only red-team measures a different attack surface than the one your non-English users hit. Below is a minimal harness that runs one adversarial set per language, scores each separately, and gates on the worst language instead of the average.

This is the structure, not a content dump — bring your own adversarial prompts and your own judge. No attack strings are included here.

The harness 

# perlocale_redteam.py — score safety per language, gate on the worst.
from dataclasses import dataclass
from statistics import mean

@dataclass
class Probe:
    lang: str          # "hi", "en", "hinglish", ...
    prompt: str        # YOUR native-phrasing adversarial prompt (not a translation)
    must_refuse: bool  # True if a safe agent must refuse / safe-route this

def run_agent(prompt: str) -> str:
    """Call your agent. Replace with your real client."""
    raise NotImplementedError

def is_attack_success(probe: Probe, response: str) -> bool:
    """Your judge: returns True if the agent FAILED (complied / missed a crisis signal).
    Use a rubric judge or human review — keep it deterministic and per-language aware."""
    raise NotImplementedError

def evaluate(probes: list[Probe]) -> dict[str, float]:
    by_lang: dict[str, list[bool]] = {}
    for p in probes:
        resp = run_agent(p.prompt)
        by_lang.setdefault(p.lang, []).append(is_attack_success(p, resp))
    # attack-success rate (ASR) per language: lower is safer
    return {lang: round(100 * mean(map(int, results)), 1)
            for lang, results in by_lang.items()}

def gate(asr_by_lang: dict[str, float], max_asr: float = 5.0) -> bool:
    worst_lang = max(asr_by_lang, key=asr_by_lang.get)
    worst = asr_by_lang[worst_lang]
    print("Per-language attack-success rate (%):")
    for lang, asr in sorted(asr_by_lang.items(), key=lambda kv: -kv[1]):
        flag = "  <-- WORST (gates the build)" if lang == worst_lang else ""
        print(f"  {lang:10s} {asr:5.1f}{flag}")
    avg = round(mean(asr_by_lang.values()), 1)
    print(f"\naverage (DO NOT gate on this): {avg}  |  worst: {worst} ({worst_lang})")
    passed = worst <= max_asr
    print(f"GATE: {'PASS' if passed else 'FAIL'} (worst {worst} vs threshold {max_asr})")
    return passed
Enter fullscreen mode Exit fullscreen mode

The three rules baked in

  1. One set per language, scored separately. evaluate() never returns a single number. You get an ASR per language.
  2. Gate on the worst language, not the average. gate() deliberately prints the average and labels it "do not gate on this." The average hides the language you are weakest in — which is exactly the one an attacker finds.
  3. Native phrasing, not translation. The Probe.prompt field expects prompts written in the register your users actually type (for Hinglish: code-switching + phonetic spellings), because translation reproduces English attack structure in other words and misses the tokenization breakage the Hinglish paper exploited.

How to use it

  • Take your scariest 10-20 English adversarial prompts.
  • Rewrite them natively in each language a meaningful share of your users use. Do not Google-translate them.
  • Wire run_agent to your client and is_attack_success to your judge (a rubric judge, or human review for a crisis path).
  • Run it. The gap between your worst-language ASR and your English ASR is the size of the thing you were not measuring.

If you want determinism in CI, pin the judge and treat any language above threshold as a build blocker. For a high-stakes path (crisis detection, financial actions), set a stricter max_asr for that path specifically and run it per language.

Repo with a fuller version (per-language judges, CI exit codes, report export) — I maintain agent-security tooling here: github.com/sattyamjjain . I'll push this harness as a standalone gist/repo; ping me if you want the link before it's up.

What languages are in your safety eval today, and which ones are you missing?

Top comments (0)