<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ali Ahmed</title>
    <description>The latest articles on DEV Community by Ali Ahmed (@quantumbreakz).</description>
    <link>https://dev.to/quantumbreakz</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4248119%2F4e211ba2-dedf-4824-ba03-bf5ecda40ce0.jpg</url>
      <title>DEV Community: Ali Ahmed</title>
      <link>https://dev.to/quantumbreakz</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/quantumbreakz"/>
    <language>en</language>
    <item>
      <title>Error budgets and burn rates explained with code: what 14.4 means, and a maintenance-window bug</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:50:11 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/error-budgets-and-burn-rates-explained-with-code-what-144-means-and-a-maintenance-window-bug-dnd</link>
      <guid>https://dev.to/quantumbreakz/error-budgets-and-burn-rates-explained-with-code-what-144-means-and-a-maintenance-window-bug-dnd</guid>
      <description>&lt;p&gt;Video: &lt;a href="https://aliahmed.si/videos/slo-burn-rate.mp4" rel="noopener noreferrer"&gt;Error budget and burn rate, visualised&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;An &lt;strong&gt;error budget&lt;/strong&gt; is the amount of failure your SLO allows: a 99.9% target over 30 days allows 0.1% of that time, which is &lt;strong&gt;43.2 minutes&lt;/strong&gt;. &lt;strong&gt;Burn rate&lt;/strong&gt; is how fast you're spending it: a burn rate of 1 uses the whole budget exactly by the end of the window, and a burn rate of &lt;strong&gt;14.4 sustained for one hour spends 2% of a 30-day budget&lt;/strong&gt;, which is the usual threshold for paging someone. Below is how I compute both in &lt;strong&gt;Kestrel&lt;/strong&gt;, a small synthetic-monitoring console I built (&lt;a href="https://github.com/QuantumBreakz/Kestrel" rel="noopener noreferrer"&gt;source&lt;/a&gt;), and a mismatch I found where the two calculations treat planned maintenance differently.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is an SLO, in numbers?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;SLI (indicator):&lt;/strong&gt; what you measure. In Kestrel, the fraction of HTTP checks that succeeded.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;SLO (objective):&lt;/strong&gt; the target for it over a window, such as 99.9% over 30 days.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Error budget:&lt;/strong&gt; &lt;code&gt;1 − target&lt;/code&gt;, as a share of the window. 0.1% of 30 days is 43.2 minutes.&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Target&lt;/th&gt;
&lt;th&gt;Budget over 30 days&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;99%&lt;/td&gt;
&lt;td&gt;7 h 12 min&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;99.5%&lt;/td&gt;
&lt;td&gt;3 h 36 min&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;99.9%&lt;/td&gt;
&lt;td&gt;43.2 min&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;99.95%&lt;/td&gt;
&lt;td&gt;21.6 min&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;99.99%&lt;/td&gt;
&lt;td&gt;4.3 min&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  How do you compute budget spent?
&lt;/h2&gt;

&lt;p&gt;Kestrel takes the checks inside the window, drops the ones that ran during &lt;strong&gt;planned maintenance&lt;/strong&gt;, and measures availability as successful checks ÷ checks. Maintenance time is also taken out of the window itself, so the budget isn't charged for downtime you announced:&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```ts title="slo.ts (core of evaluateSlo)"&lt;br&gt;
const failures = considered.filter((run) =&amp;gt; !run.ok).length;&lt;br&gt;
const availability = (considered.length - failures) / considered.length;&lt;/p&gt;

&lt;p&gt;const windowMs = evaluation.end - evaluation.start - excusedTime(windows, evaluation);&lt;br&gt;
const allowedDowntimeMs = windowMs * (1 - monitor.sloTarget);&lt;br&gt;
const observedDowntimeMs = windowMs * (1 - availability);&lt;/p&gt;

&lt;p&gt;const budgetSpent = allowedDowntimeMs &amp;gt; 0 ? observedDowntimeMs / allowedDowntimeMs : 0;&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


`budgetSpent` above 1 means the SLO is breached. One detail matters in `excusedTime`: overlapping maintenance windows are **merged before they're summed**, so a minute covered by two announcements isn't excused twice. Without that, two overlapping two-hour windows would excuse four hours.

## What is burn rate, and why 14.4?

Burn rate compares the recent failure rate with the failure rate the SLO allows:



```ts title="Burn rate"
const failureRate = failedChecks / totalChecks;
const allowedFailureRate = 1 - monitor.sloTarget;
const burnRate = failureRate / allowedFailureRate;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;At burn rate 1, you finish the window with exactly zero budget left. At 10, you'd use 30 days of budget in 3 days.&lt;/p&gt;

&lt;p&gt;The 14.4 comes from asking "how much budget can one bad hour cost before someone should wake up?" A 30-day window has 720 hours. Burning at 14.4× for 1 hour uses 14.4 ÷ 720 = &lt;strong&gt;2% of the budget&lt;/strong&gt;. Google's SRE workbook recommends paging pairs along these lines:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Burn rate&lt;/th&gt;
&lt;th&gt;Measured over&lt;/th&gt;
&lt;th&gt;Budget used&lt;/th&gt;
&lt;th&gt;Action&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;14.4&lt;/td&gt;
&lt;td&gt;1 hour&lt;/td&gt;
&lt;td&gt;2%&lt;/td&gt;
&lt;td&gt;Page&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;6 hours&lt;/td&gt;
&lt;td&gt;5%&lt;/td&gt;
&lt;td&gt;Page&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;3 days&lt;/td&gt;
&lt;td&gt;10%&lt;/td&gt;
&lt;td&gt;Ticket&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A short window catches outages quickly; a long one catches slow leaks that never look dramatic in any single hour. Real alerting also checks a much shorter window (say five minutes) at the same time, so the alert stops soon after the problem does.&lt;/p&gt;
&lt;h2&gt;
  
  
  What was the maintenance-window bug?
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;evaluateSlo&lt;/code&gt; excludes checks during planned maintenance. &lt;code&gt;fastBurnRate&lt;/code&gt;, the short-window function for paging, &lt;strong&gt;doesn't take maintenance windows at all&lt;/strong&gt;. During a planned one-hour migration where every check fails, the 30-day report correctly charges nothing, while the burn rate reads 1,000 for a 99.9% target, far above the 14.4 threshold. An alert built on it would page someone for maintenance everyone already knew about.&lt;/p&gt;

&lt;p&gt;It hadn't caused a page yet because nothing in the UI calls &lt;code&gt;fastBurnRate&lt;/code&gt;. That's the moment to fix it, before alerting is built on it. The fix is to filter with the same rule in both places:&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```ts title="Same exclusion for both calculations"&lt;br&gt;
export function fastBurnRate(monitor: Monitor, runs: CheckRun[], windows: MaintenanceWindow[], now: number, lookbackMs: number): number {&lt;br&gt;
  const recent = runs.filter((run) =&amp;gt; run.startedAt &amp;gt;= now - lookbackMs &amp;amp;&amp;amp; run.startedAt &amp;lt;= now &amp;amp;&amp;amp; !isExcused(run.startedAt, windows));&lt;br&gt;
  if (recent.length === 0) return 0;&lt;/p&gt;

&lt;p&gt;const failureRate = recent.filter((run) =&amp;gt; !run.ok).length / recent.length;&lt;br&gt;
  const allowedFailureRate = 1 - monitor.sloTarget;&lt;br&gt;
  if (allowedFailureRate &amp;lt;= 0) return failureRate &amp;gt; 0 ? Infinity : 0;&lt;br&gt;
  return failureRate / allowedFailureRate;&lt;br&gt;
}&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


And a test that pins the behaviour: a window where every check fails inside maintenance must return a burn rate of 0.

## What are the limits of check-based availability?

Counting checks treats each one equally. If one region checks every 30 seconds and another every 5 minutes, the fast region dominates the number. Either check every region at the same interval, or weight each check by the time it represents. And a synthetic check only measures what it requests: a green health endpoint says nothing about a broken checkout page.

## Summary

* Error budget = `(1 − target) × window`. 99.9% over 30 days = 43.2 minutes.
* Budget spent = observed downtime ÷ allowed downtime; above 1 means breached.
* Burn rate = failure rate ÷ allowed failure rate. 14.4 for an hour spends 2% of a 30-day budget.
* **Apply the same exclusions** (maintenance, test traffic) to every calculation that feeds an alert.

For more on what runs in production and why, see [what actually gets deployed to production](https://aliahmed.si/blog/what-gets-deployed-to-production/) and [labels vs annotations in Prometheus ServiceMonitors](https://aliahmed.si/blog/servicemonitor-labels-vs-annotations/).

Latency SLOs need percentiles, and those have their own trap: [why you can't average p95 latencies](https://aliahmed.si/blog/latency-percentiles-p95-averaging/).

---

Written by Ali Ahmed · [Original article](https://aliahmed.si/blog/slo-error-budget-burn-rate/)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>sre</category>
      <category>monitoring</category>
      <category>typescript</category>
      <category>reliability</category>
    </item>
    <item>
      <title>Single agent or multi-agent? What I learned building a multi-agent financial intelligence system</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:48:36 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/single-agent-or-multi-agent-what-i-learned-building-a-multi-agent-financial-intelligence-system-45j7</link>
      <guid>https://dev.to/quantumbreakz/single-agent-or-multi-agent-what-i-learned-building-a-multi-agent-financial-intelligence-system-45j7</guid>
      <description>&lt;p&gt;&lt;strong&gt;Start with one agent and good tools.&lt;/strong&gt; Split into several agents only when the work has &lt;strong&gt;parts that need different instructions, tools or permissions&lt;/strong&gt;, parts that can &lt;strong&gt;run in parallel&lt;/strong&gt;, or a step that should &lt;strong&gt;independently check&lt;/strong&gt; another's output. Each extra agent adds coordination, latency, cost and new ways to fail, so it has to earn its place. I built &lt;strong&gt;Cuttlefish&lt;/strong&gt;, a multi-agent financial intelligence platform (&lt;a href="https://github.com/QuantumBreakz/Cuttlefish-backend" rel="noopener noreferrer"&gt;backend source&lt;/a&gt;), with seven agent types and sequential, parallel and hybrid workflows. Here's what that taught me about when it's worth it and how to make it safe.&lt;/p&gt;

&lt;h2&gt;
  
  
  What does a multi-agent system actually add?
&lt;/h2&gt;

&lt;p&gt;In Cuttlefish, separate agents handle building and scenario analysis, market signals, permits, predictions, compliance, refactoring, and a &lt;strong&gt;meta-auditor&lt;/strong&gt; that reviews the combined result. A workflow lists which agents run, and in what mode:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Sequential:&lt;/strong&gt; each agent sees the context built by the ones before it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Parallel:&lt;/strong&gt; independent agents run at the same time with &lt;code&gt;asyncio.gather&lt;/code&gt;, cutting wall-clock time.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hybrid:&lt;/strong&gt; a mix of both.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The real benefits were &lt;strong&gt;separation&lt;/strong&gt; (each agent has a short, focused prompt and only the tools it needs) and &lt;strong&gt;parallelism&lt;/strong&gt; (independent analyses don't wait for each other). Neither needs autonomous agents negotiating with each other; a fixed workflow did the job, and was much easier to debug.&lt;/p&gt;

&lt;h2&gt;
  
  
  When is one agent better?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;The task is &lt;strong&gt;one line of reasoning&lt;/strong&gt;: answer a question, fix a bug, fill a form.&lt;/li&gt;
&lt;li&gt;The steps need &lt;strong&gt;all the same context&lt;/strong&gt;. Splitting means copying context between agents, paying for it again, and losing details in the handoff.&lt;/li&gt;
&lt;li&gt;You can't yet &lt;strong&gt;evaluate&lt;/strong&gt; one agent reliably. Several agents multiply the cases you have to test; see &lt;a href="https://aliahmed.si/blog/evaluating-ai-agents/" rel="noopener noreferrer"&gt;how to evaluate AI agents&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Most "multi-agent" problems are solved by one agent with well-designed &lt;a href="https://aliahmed.si/blog/llm-tool-calling-explained/" rel="noopener noreferrer"&gt;tools&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Lesson 1: every agent needs a timeout
&lt;/h2&gt;

&lt;p&gt;One slow agent stalls a sequential workflow and holds a parallel one open. Cuttlefish gives each agent a default timeout (25 seconds, configurable per agent type) and records a timeout as an error entry in the result, then moves on to the next agent. Decide up front what the workflow does without that agent's output: continue with a gap clearly marked, or fail the task.&lt;/p&gt;

&lt;h2&gt;
  
  
  Lesson 2: checks must fail closed
&lt;/h2&gt;

&lt;p&gt;Reviewing the code later, I found the workflow treated a &lt;strong&gt;missing meta-auditor as a pass&lt;/strong&gt;: if the auditor agent wasn't registered, the audit returned &lt;code&gt;passed: True&lt;/code&gt; with the message "No meta-auditor available, skipping audit". A misconfigured deployment would therefore mark every result as audited. A verification step that passes when it can't run isn't a verification step. It should &lt;strong&gt;fail closed&lt;/strong&gt;: no auditor, no "audited" status. It's the same rule I apply to &lt;a href="https://aliahmed.si/blog/prompt-injection-scraped-data-llm-outreach/" rel="noopener noreferrer"&gt;send gates for LLM-written emails&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The audit itself should also be &lt;strong&gt;independent&lt;/strong&gt;: a reviewer that only re-reads the same agents' conclusions tends to agree with them. Checks in code (do the numbers trace back to sources? are required fields present?) are worth more than another model's opinion; see &lt;a href="https://aliahmed.si/blog/stop-llm-inventing-numbers/" rel="noopener noreferrer"&gt;how I stop an LLM from inventing numbers&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Lesson 3: log every agent action
&lt;/h2&gt;

&lt;p&gt;When a final number looks wrong, you need to know which agent produced which input. Cuttlefish logs every agent action to a hash-chained log. That log had three bugs of its own, including entries that could never be verified and parallel agents forking the chain: &lt;a href="https://aliahmed.si/blog/ai-agent-audit-log-hash-chain/" rel="noopener noreferrer"&gt;the fixes are here&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  A decision checklist
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;If yes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Do parts need different tools or permissions?&lt;/td&gt;
&lt;td&gt;Separate agents (least privilege)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can parts run independently?&lt;/td&gt;
&lt;td&gt;Parallel agents save time&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Should one step check another independently?&lt;/td&gt;
&lt;td&gt;A reviewer, ideally checks in code&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Does every step need the same full context?&lt;/td&gt;
&lt;td&gt;Keep one agent&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Can you evaluate a single agent reliably yet?&lt;/td&gt;
&lt;td&gt;If not, don't add more&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;One agent with good tools first.&lt;/li&gt;
&lt;li&gt;Split for &lt;strong&gt;permissions, parallelism or independent review&lt;/strong&gt;, using fixed workflows before autonomous ones.&lt;/li&gt;
&lt;li&gt;Timeouts on every agent, checks that &lt;strong&gt;fail closed&lt;/strong&gt;, and a log of every action.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Related: &lt;a href="https://aliahmed.si/blog/what-is-model-context-protocol/" rel="noopener noreferrer"&gt;what MCP is&lt;/a&gt;, &lt;a href="https://aliahmed.si/blog/what-is-an-llm-gateway-litellm/" rel="noopener noreferrer"&gt;LLM gateways&lt;/a&gt;, and &lt;a href="https://aliahmed.si/blog/agent-tool-error-messages/" rel="noopener noreferrer"&gt;tool errors agents can act on&lt;/a&gt;. More about Cuttlefish and my other work is on the &lt;a href="https://aliahmed.si/projects/" rel="noopener noreferrer"&gt;projects page&lt;/a&gt;.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/single-agent-vs-multi-agent/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>aiagents</category>
      <category>llms</category>
      <category>aiengineering</category>
      <category>python</category>
    </item>
    <item>
      <title>How to cut LLM API costs without making the product worse: caching, routing, batching and context</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:44:36 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/how-to-cut-llm-api-costs-without-making-the-product-worse-caching-routing-batching-and-context-17o5</link>
      <guid>https://dev.to/quantumbreakz/how-to-cut-llm-api-costs-without-making-the-product-worse-caching-routing-batching-and-context-17o5</guid>
      <description>&lt;p&gt;Most LLM bills are dominated by &lt;strong&gt;input tokens&lt;/strong&gt;: long system prompts, conversation history and retrieved documents sent again on every request. The levers that cut cost without hurting quality, roughly in order of effort, are: &lt;strong&gt;measure cost per feature&lt;/strong&gt;, &lt;strong&gt;send less context&lt;/strong&gt;, use &lt;strong&gt;prompt caching&lt;/strong&gt; for repeated prefixes, &lt;strong&gt;route&lt;/strong&gt; easy requests to cheaper models, use &lt;strong&gt;batch APIs&lt;/strong&gt; for anything that isn't interactive, and &lt;strong&gt;cap output length and retries&lt;/strong&gt;. Before changing anything, run an evaluation set so you can prove quality didn't drop. I've worked on this from the gateway side as a contributor to LiteLLM, and in the products I build.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: Measure cost per feature first
&lt;/h2&gt;

&lt;p&gt;A total monthly bill doesn't tell you what to fix. Tag every request with the &lt;strong&gt;feature&lt;/strong&gt; and &lt;strong&gt;prompt version&lt;/strong&gt;, and record input tokens, output tokens, cached tokens, model and latency. A gateway such as LiteLLM can do this centrally. One feature, often a chat with a long history or an agent loop, usually accounts for most of the spend.&lt;/p&gt;

&lt;p&gt;Make sure the cost numbers are right. In LiteLLM's auto-router, savings were reported as $0.00 when two deployments of the same model had different prices, because it compared model names instead of deployments. I reported it with a fix and regression tests in &lt;a href="https://github.com/BerriAI/litellm/pull/38834" rel="noopener noreferrer"&gt;#38834&lt;/a&gt;, and the maintainers resolved it upstream; the write-up is &lt;a href="https://aliahmed.si/blog/llm-router-savings-deployment-id/" rel="noopener noreferrer"&gt;why model name is not identity&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: Send less context
&lt;/h2&gt;

&lt;p&gt;The cheapest token is one you don't send:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Trim history:&lt;/strong&gt; keep the last few turns plus a short summary, not the whole conversation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Retrieve less, retrieve better:&lt;/strong&gt; three good chunks beat ten mediocre ones, for cost &lt;strong&gt;and&lt;/strong&gt; quality. Measure with &lt;a href="https://aliahmed.si/blog/rag-retrieval-evaluation-metrics/" rel="noopener noreferrer"&gt;recall@k and MRR&lt;/a&gt; before and after.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Shorten tool results:&lt;/strong&gt; an agent that pastes a 20,000-token API response into its context pays for it on every following step. Return the fields the model needs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Compress carefully.&lt;/strong&gt; Context compression can save a lot, but it must never compress what the agent explicitly asked for. That exact bug made Claude Code loop forever behind LiteLLM: &lt;a href="https://aliahmed.si/blog/litellm-context-compression-bug/" rel="noopener noreferrer"&gt;the context-compression fix&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Step 3: Use prompt caching
&lt;/h2&gt;

&lt;p&gt;If many requests start with the same long prefix (system prompt, tool definitions, a document), &lt;strong&gt;prompt caching&lt;/strong&gt; lets the provider reuse it. Cached input is billed at a fraction of the normal price; on Anthropic's API, cache reads cost a tenth of the base input price, while OpenAI applies caching automatically at a discount. Check your provider's current pricing. To benefit:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Put &lt;strong&gt;stable content first&lt;/strong&gt; (instructions, tools, documents) and variable content (the user's message) last.&lt;/li&gt;
&lt;li&gt;Don't put timestamps or request IDs in the prefix, or nothing will ever match.&lt;/li&gt;
&lt;li&gt;Watch the &lt;strong&gt;cache hit rate&lt;/strong&gt; in your logs.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Step 4: Route requests to the right model
&lt;/h2&gt;

&lt;p&gt;Many requests don't need the largest model: classification, extraction, short rewrites, simple questions. A router sends those to a smaller, cheaper model and keeps the large one for hard requests. Rules (by feature or input length) are a fine start; a classifier-based router comes later. Measure quality per route, since a cheaper model that fails more often costs more in retries and support.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 5: Batch anything that isn't interactive
&lt;/h2&gt;

&lt;p&gt;Embedding a document collection, nightly summaries, evaluation runs and back-fills don't need an answer in two seconds. &lt;strong&gt;Batch APIs&lt;/strong&gt; from major providers process requests asynchronously, typically within 24 hours, at about &lt;strong&gt;half the price&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 6: Cap output and retries
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Set &lt;code&gt;max_tokens&lt;/code&gt;&lt;/strong&gt; per feature. Output tokens usually cost several times more than input tokens.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Ask for structured output&lt;/strong&gt; instead of prose when code consumes the result; it's shorter and easier to validate. See &lt;a href="https://aliahmed.si/blog/llm-structured-output-validation/" rel="noopener noreferrer"&gt;reliable JSON from LLMs&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Limit retries and agent steps.&lt;/strong&gt; An agent stuck in a loop is the most expensive bug in AI engineering. Errors that tell the agent what to do next prevent many loops; see &lt;a href="https://aliahmed.si/blog/agent-tool-error-messages/" rel="noopener noreferrer"&gt;tool errors for AI agents&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What about semantic caching?
&lt;/h2&gt;

&lt;p&gt;Caching answers for &lt;strong&gt;similar&lt;/strong&gt; questions (by embedding similarity) sounds attractive, but it returns one user's answer to a different question that merely looks similar. Use it only where answers don't depend on the user or on time, and with a strict similarity threshold. Exact-match caching, keyed by a hash of the prompt and context, is safe and simple, as in my &lt;a href="https://aliahmed.si/blog/offline-rag-chatbot-ollama/" rel="noopener noreferrer"&gt;offline RAG chatbot&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The checklist
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Tag and measure cost per feature.&lt;/li&gt;
&lt;li&gt;Build an evaluation set, so quality is checked after every change.&lt;/li&gt;
&lt;li&gt;Trim context and tool results.&lt;/li&gt;
&lt;li&gt;Order prompts for caching.&lt;/li&gt;
&lt;li&gt;Route easy requests to smaller models.&lt;/li&gt;
&lt;li&gt;Batch offline work.&lt;/li&gt;
&lt;li&gt;Cap output tokens, retries and agent steps.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;More about my LiteLLM contributions and AI work is on the &lt;a href="https://aliahmed.si/" rel="noopener noreferrer"&gt;home page&lt;/a&gt; and the &lt;a href="https://aliahmed.si/projects/" rel="noopener noreferrer"&gt;projects page&lt;/a&gt;.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/reduce-llm-api-costs/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>llms</category>
      <category>aiengineering</category>
      <category>litellm</category>
      <category>cost</category>
    </item>
    <item>
      <title>RAG chunking strategies: fixed size, by heading, and what actually changes retrieval quality</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:41:25 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/rag-chunking-strategies-fixed-size-by-heading-and-what-actually-changes-retrieval-quality-3k0f</link>
      <guid>https://dev.to/quantumbreakz/rag-chunking-strategies-fixed-size-by-heading-and-what-actually-changes-retrieval-quality-3k0f</guid>
      <description>&lt;p&gt;&lt;strong&gt;Chunking&lt;/strong&gt; is how you split documents into the pieces that get embedded and retrieved, and it decides what a RAG system &lt;strong&gt;can&lt;/strong&gt; find: if the answer is cut in half, or buried in a chunk about something else, no model will recover it. Start with &lt;strong&gt;structure-aware chunks&lt;/strong&gt; (split by headings, then by paragraphs up to a size limit), &lt;strong&gt;keep the section heading in every chunk&lt;/strong&gt;, and use fixed-size chunks with overlap only for text without structure. Then choose the size by &lt;strong&gt;measuring recall\@k on your own questions&lt;/strong&gt;, not by copying a default.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fixed-size chunks with overlap
&lt;/h2&gt;

&lt;p&gt;The simplest method: cut every N characters (or tokens), with some overlap so a sentence on a boundary appears in both chunks. It works on any text and needs no parsing. In my &lt;a href="https://aliahmed.si/blog/offline-rag-chatbot-ollama/" rel="noopener noreferrer"&gt;offline RAG chatbot&lt;/a&gt;, 600 characters with 200 overlap worked well for a 7B local model.&lt;/p&gt;

&lt;p&gt;The weaknesses: chunks cut through the middle of a table, list or argument, and a chunk from the middle of a section loses its context ("The limit is 30 days" — the limit for what?).&lt;/p&gt;

&lt;h2&gt;
  
  
  Structure-aware chunking by heading
&lt;/h2&gt;

&lt;p&gt;Most real documents (policies, docs, contracts, wikis) have headings, and a section is usually about one thing. Split on headings first, then split long sections at paragraph breaks, and &lt;strong&gt;repeat the heading&lt;/strong&gt; in each piece:&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```python title="chunk.py"&lt;br&gt;
def chunk_by_heading(markdown: str, max_chars: int = 1200) -&amp;gt; list[str]:&lt;br&gt;
    sections, current = [], []&lt;br&gt;
    for line in markdown.splitlines():&lt;br&gt;
        if line.startswith("#") and current:&lt;br&gt;
            sections.append("\n".join(current))&lt;br&gt;
            current = []&lt;br&gt;
        current.append(line)&lt;br&gt;
    if current:&lt;br&gt;
        sections.append("\n".join(current))&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;chunks = []
for section in sections:
    heading = section.splitlines()[0] if section.startswith("#") else ""
    while len(section) &amp;gt; max_chars:
        cut = section.rfind("\n\n", 0, max_chars)
        cut = cut if cut &amp;gt; 0 else max_chars
        chunks.append(section[:cut].strip())
        section = (heading + "\n" + section[cut:].strip()) if heading else section[cut:].strip()
    chunks.append(section.strip())
return [c for c in chunks if c]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


Long sections are cut at the last paragraph break before the limit, and every continuation starts with its heading, so "The limit is 30 days" arrives as "## Refund policy … The limit is 30 days". Convert PDFs and HTML to Markdown first so headings survive.

## Add context to every chunk

Beyond the heading, prepend what a reader would need: the **document title**, and for some corpora the date or product version. Some teams go further and have an LLM write a one-sentence summary of where each chunk sits in its document, embedded along with the chunk. It costs one LLM call per chunk at indexing time, so measure whether it helps.

Store **metadata** with each chunk too (source, section, date, access level). It lets you filter before searching ("only 2026 policies"), cite sources in answers, and enforce permissions, so users only retrieve documents they're allowed to read.

## Special cases

* **Tables:** keep a table whole if it fits, and repeat the header row when splitting. A row without its header is a list of meaningless numbers.
* **Code:** split by function or class, not by character count.
* **Q\&amp;amp;A and FAQs:** one question with its answer per chunk.
* **Very short documents:** don't split them at all.

## How do you choose the size?

Small chunks are precise but lose context; large chunks keep context but dilute the embedding, so similarity search gets less precise, and they use more of the model's context. Don't guess:

1. Build a [golden set of 50–100 real questions](https://aliahmed.si/blog/rag-retrieval-evaluation-metrics/) with the passages that answer them.
2. Index the corpus with two or three strategies and sizes.
3. Compare **recall\@k** and **MRR**, then check answer groundedness for the best two.

Re-run it when the document mix changes. Chunking that suits policy documents may be wrong for support tickets.

## Summary

* Split by **structure** first, size second.
* Keep the **heading and title** in every chunk.
* Store **metadata** for filtering, citations and permissions.
* Choose sizes by **measuring recall**, not by default.

Related: [embeddings and vector search](https://aliahmed.si/blog/embeddings-vector-search-explained/), [evaluating RAG](https://aliahmed.si/blog/rag-retrieval-evaluation-metrics/), and [reliable structured output from LLMs](https://aliahmed.si/blog/llm-structured-output-validation/). More on the [projects page](https://aliahmed.si/projects/).

---

Written by Ali Ahmed · [Original article](https://aliahmed.si/blog/rag-chunking-strategies/)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>rag</category>
      <category>llms</category>
      <category>aiengineering</category>
      <category>python</category>
    </item>
    <item>
      <title>How passport MRZ check digits work, with code to validate them</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:39:13 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/how-passport-mrz-check-digits-work-with-code-to-validate-them-1378</link>
      <guid>https://dev.to/quantumbreakz/how-passport-mrz-check-digits-work-with-code-to-validate-them-1378</guid>
      <description>&lt;p&gt;Video: &lt;a href="https://aliahmed.si/videos/mrz-check-digit.mp4" rel="noopener noreferrer"&gt;Computing a passport MRZ check digit, step by step&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The &lt;strong&gt;machine-readable zone (MRZ)&lt;/strong&gt; is the block of uppercase letters, digits and &lt;code&gt;&amp;lt;&lt;/code&gt; chevrons at the bottom of a passport data page. It repeats the key fields in a fixed format defined by &lt;strong&gt;ICAO Document 9303&lt;/strong&gt;, and it carries &lt;strong&gt;check digits&lt;/strong&gt;: each important field is followed by one digit computed from it with weights &lt;strong&gt;7, 3, 1&lt;/strong&gt;. A KYC pipeline uses them to catch OCR mistakes and some tampering. If a single character in the document number is misread, its check digit almost always stops matching. Here's how the algorithm works, with tested Python.&lt;/p&gt;

&lt;h2&gt;
  
  
  What does an MRZ look like?
&lt;/h2&gt;

&lt;p&gt;A passport (format &lt;strong&gt;TD3&lt;/strong&gt;) has two lines of 44 characters. This is ICAO's own specimen, for the fictional state "Utopia":&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```text title="TD3 specimen (ICAO 9303)"&lt;br&gt;
P&amp;lt;UTOERIKSSON&amp;lt;&amp;lt;ANNA&amp;lt;MARIA&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&lt;br&gt;
L898902C36UTO7408122F1204159ZE184226B&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;10&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


The second line holds the data, in fixed positions:

| Positions | Field                         | In the specimen  |
| --------- | ----------------------------- | ---------------- |
| 1–9       | Document number               | `L898902C3`      |
| 10        | Check digit for the number    | `6`              |
| 11–13     | Nationality                   | `UTO`            |
| 14–19     | Date of birth (YYMMDD)        | `740812`         |
| 20        | Check digit for date of birth | `2`              |
| 21        | Sex                           | `F`              |
| 22–27     | Expiry date (YYMMDD)          | `120415`         |
| 28        | Check digit for expiry        | `9`              |
| 29–42     | Optional data                 | `ZE184226B&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;` |
| 43        | Check digit for optional data | `1`              |
| 44        | Composite check digit         | `0`              |

ID cards use other layouts: **TD1** has three lines of 30 characters and **TD2** two lines of 36, but the check digit algorithm is the same.

## How is a check digit calculated?

1. Convert each character to a number: digits keep their value, letters `A`–`Z` become **10–35**, and the filler `&amp;lt;` is **0**.
2. Multiply each value by a repeating weight of **7, 3, 1, 7, 3, 1, …**
3. Add the products and take the remainder after dividing by **10**.



```python title="mrz.py"
def check_digit(field: str) -&amp;gt; int:
    weights = (7, 3, 1)
    total = 0
    for i, ch in enumerate(field):
        if ch.isdigit():
            value = int(ch)
        elif ch.isalpha():
            value = ord(ch.upper()) - ord("A") + 10
        else:
            value = 0
        total += value * weights[i % 3]
    return total % 10
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;For the specimen, &lt;code&gt;check_digit("L898902C3")&lt;/code&gt; returns 6, &lt;code&gt;check_digit("740812")&lt;/code&gt; returns 2, and &lt;code&gt;check_digit("120415")&lt;/code&gt; returns 9, matching the printed digits.&lt;/p&gt;
&lt;h2&gt;
  
  
  What is the composite check digit?
&lt;/h2&gt;

&lt;p&gt;The last digit of a TD3 line covers &lt;strong&gt;several fields at once&lt;/strong&gt;: the document number with its check digit (positions 1–10), the date of birth with its digit (14–20), and the expiry date, optional data and their digits (22–43), joined together:&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```python title="Validate the composite digit"&lt;br&gt;
line2 = "L898902C36UTO7408122F1204159ZE184226B&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;10"&lt;br&gt;
composite = line2[0:10] + line2[13:20] + line2[21:43]&lt;br&gt;
assert check_digit(composite) == int(line2[43])&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


The composite catches errors that individual digits can miss, such as two misreads that happen to cancel out within one field.

## Why does this matter for OCR?

MRZ text is printed in **OCR-B**, a font designed for machines, but real photos still produce confusions: `0` and `O`, `1` and `I`, `8` and `B`, `5` and `S`, and dropped or doubled chevrons. Check digits turn "the OCR might be wrong" into something you can test:

* **Reject or retry** a reading whose digits don't validate, instead of storing a wrong passport number.
* **Repair using position rules.** Dates and check digits must be numeric, so an `O` in a date is a `0`. The document number can contain letters, so it needs the check digit to decide.
* **Cross-check against the visual zone.** The MRZ name and dates should match what OCR reads from the printed fields above. A mismatch means either an OCR error or a tampered page, and both need review. This is one of the stages in [how KYC identity verification works](https://aliahmed.si/blog/how-kyc-identity-verification-works/).

## What check digits don't prove

Check digits detect **accidental errors**. They are not security: anyone forging a document can compute valid digits with the same ten lines of code. Proving a passport is genuine takes authenticity checks or, better, reading the **NFC chip** in e-passports, whose data is cryptographically signed by the issuing country.

## Related

* [Why OCR struggles with Arabic, Urdu and other non-Latin scripts](https://aliahmed.si/blog/ocr-non-latin-scripts-arabic-urdu/)
* [Face match thresholds: false accepts vs false rejects](https://aliahmed.si/blog/face-match-threshold-far-frr/)
* [Face liveness detection explained](https://aliahmed.si/blog/face-liveness-detection-explained/)
* [A real-time KYC card verification pipeline over MQTT](https://aliahmed.si/blog/mqtt-kyc-verification-pipeline/)

I build computer vision and OCR systems for identity verification; more on the [home page](https://aliahmed.si/).

Check digits catch errors; the NFC chip proves authenticity: [how e-passport chip verification works](https://aliahmed.si/blog/nfc-epassport-chip-verification/).

---

Written by Ali Ahmed · [Original article](https://aliahmed.si/blog/passport-mrz-check-digits/)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>kyc</category>
      <category>ocr</category>
      <category>python</category>
      <category>computervision</category>
    </item>
    <item>
      <title>Why OCR struggles with Arabic, Urdu and other non-Latin scripts, and what helps</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:36:02 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/why-ocr-struggles-with-arabic-urdu-and-other-non-latin-scripts-and-what-helps-49jc</link>
      <guid>https://dev.to/quantumbreakz/why-ocr-struggles-with-arabic-urdu-and-other-non-latin-scripts-and-what-helps-49jc</guid>
      <description>&lt;p&gt;OCR models are mostly trained and benchmarked on &lt;strong&gt;Latin text in straight lines&lt;/strong&gt;. Identity documents from 100+ countries break those assumptions: &lt;strong&gt;Arabic&lt;/strong&gt; letters join and change shape by position, &lt;strong&gt;Urdu&lt;/strong&gt; is usually written in &lt;strong&gt;Nastaliq&lt;/strong&gt;, whose words slope down diagonally, text runs &lt;strong&gt;right to left&lt;/strong&gt; while numbers run left to right, some documents use &lt;strong&gt;Eastern Arabic digits&lt;/strong&gt;, and &lt;strong&gt;Thai&lt;/strong&gt; and &lt;strong&gt;Chinese&lt;/strong&gt; don't put spaces between words. What helps most is identifying the script first, routing to a model trained on it, and normalising the output before validating it. This is the part of KYC I worked on at ShuftiPro, building multilingual OCR for identity documents.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why is Arabic script hard for OCR?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Letters connect.&lt;/strong&gt; Arabic is cursive in print as well as handwriting, so you can't segment a word into separate characters the way you can with Latin text.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One letter, up to four shapes.&lt;/strong&gt; Most letters have isolated, initial, medial and final forms. The model has to recognise all of them as the same character.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dots carry meaning.&lt;/strong&gt; Several letters differ only in the number and position of dots (ب ت ث). A speck of dust or a worn laminate can change the letter.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Optional diacritics.&lt;/strong&gt; Short vowels are usually omitted, so the same spelling can have several pronunciations, and transliterating a name into Latin letters is ambiguous.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What makes Urdu Nastaliq harder still?
&lt;/h2&gt;

&lt;p&gt;Urdu uses the Arabic alphabet with extra letters, but it's commonly typeset in &lt;strong&gt;Nastaliq&lt;/strong&gt;, a calligraphic style where each joined group of letters &lt;strong&gt;slopes from upper right to lower left&lt;/strong&gt;. There's no single horizontal baseline, letters stack and overlap, and the same word can have a different width depending on its neighbours. Models built for Naskh-style Arabic, which sits on a flat baseline, do noticeably worse on Nastaliq. Pakistani documents carry Urdu next to English, so a pipeline has to handle both on one card.&lt;/p&gt;

&lt;h2&gt;
  
  
  Right-to-left text with left-to-right numbers
&lt;/h2&gt;

&lt;p&gt;Arabic and Urdu read right to left, but numbers inside them read &lt;strong&gt;left to right&lt;/strong&gt;. Unicode stores text in &lt;strong&gt;logical order&lt;/strong&gt; (the order you'd type it) and leaves display order to the bidirectional algorithm. An OCR model that outputs characters in visual order, left to right across the image, scrambles mixed text. Dates such as &lt;code&gt;12/04/1990&lt;/code&gt; inside Urdu text are the classic failure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Eastern Arabic digits
&lt;/h2&gt;

&lt;p&gt;Many documents print digits as &lt;strong&gt;٠١٢٣٤٥٦٧٨٩&lt;/strong&gt; (Arabic-Indic) or the Persian and Urdu variants &lt;strong&gt;۰۱۲۳۴۵۶۷۸۹&lt;/strong&gt;. They're different Unicode characters from &lt;code&gt;0–9&lt;/code&gt;, so a database lookup or a date parser fails unless you normalise them:&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```python title="Normalise Eastern Arabic and Persian digits"&lt;br&gt;
EASTERN = str.maketrans("٠١٢٣٤٥٦٧٨٩۰۱۲۳۴۵۶۷۸۹", "01234567890123456789")&lt;/p&gt;

&lt;p&gt;assert "٣٥٢٠١-۱۲۳".translate(EASTERN) == "35201-123"&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


Normalise **before** validation, so a document number read in either digit set passes the same checks.

## Scripts without spaces, and very large alphabets

* **Thai, Lao, Khmer, Chinese and Japanese** don't separate words with spaces, so splitting a name into given and family names needs language knowledge, not whitespace.
* **Chinese and Japanese** have thousands of characters in common use, so rare characters in names are under-represented in training data.
* **Devanagari** (Hindi, Nepali) joins letters under a horizontal headline and forms conjuncts, which again defeats character-by-character segmentation.

## What actually helps

1. **Detect the document type and script first,** then route to a recogniser trained on that script and font. One model for everything is rarely the most accurate.
2. **Use the layout.** On a known document template, you know where each field is and which script and format to expect, so a date field can only contain a date.
3. **Prefer the MRZ when there is one.** It's Latin, fixed-format and has check digits. See [how passport MRZ check digits work](https://aliahmed.si/blog/passport-mrz-check-digits/). Use it to cross-check the native-script fields.
4. **Normalise output:** digits, Unicode normalisation (NFC), and the different code points for visually identical letters (Arabic and Persian `ي` and `ی`, for example).
5. **Train on real captures,** with glare, blur and worn cards, not just clean renders. Synthetic data helps fill the gaps for rare scripts.
6. **Measure accuracy per script and per field.** An overall accuracy number hides the script where the system fails most.

## Where this fits

OCR is one stage of [how KYC identity verification works](https://aliahmed.si/blog/how-kyc-identity-verification-works/), alongside [face matching](https://aliahmed.si/blog/face-match-threshold-far-frr/) and [liveness detection](https://aliahmed.si/blog/face-liveness-detection-explained/). For a much smaller pipeline that checks fields after extraction, see the [MQTT KYC verification project](https://aliahmed.si/blog/mqtt-kyc-verification-pipeline/). And if you're building retrieval over documents in these scripts, [offline RAG with Ollama](https://aliahmed.si/blog/offline-rag-chatbot-ollama/) covers the chunking side.

More about my computer vision and OCR work is on the [home page](https://aliahmed.si/).

Before OCR runs, the photo itself has to be usable: [preparing ID card photos for OCR with OpenCV](https://aliahmed.si/blog/document-photo-preprocessing-ocr/).

---

Written by Ali Ahmed · [Original article](https://aliahmed.si/blog/ocr-non-latin-scripts-arabic-urdu/)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>ocr</category>
      <category>computervision</category>
      <category>kyc</category>
      <category>aiengineering</category>
    </item>
    <item>
      <title>How NFC passport chip verification works: BAC, PACE, passive and chip authentication</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:33:15 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/how-nfc-passport-chip-verification-works-bac-pace-passive-and-chip-authentication-27i</link>
      <guid>https://dev.to/quantumbreakz/how-nfc-passport-chip-verification-works-bac-pace-passive-and-chip-authentication-27i</guid>
      <description>&lt;p&gt;An e-passport (look for the chip symbol on the cover) contains an &lt;strong&gt;NFC chip&lt;/strong&gt; holding the data page and the holder's photo, &lt;strong&gt;digitally signed by the issuing country&lt;/strong&gt;. A phone reads it in three steps: it &lt;strong&gt;unlocks the chip&lt;/strong&gt; with a key derived from the passport's MRZ (&lt;strong&gt;BAC&lt;/strong&gt; or the newer &lt;strong&gt;PACE&lt;/strong&gt;), &lt;strong&gt;reads the data groups&lt;/strong&gt;, and &lt;strong&gt;checks the signatures&lt;/strong&gt; back to the country's certificate (&lt;strong&gt;passive authentication&lt;/strong&gt;). A further check, &lt;strong&gt;chip or active authentication&lt;/strong&gt;, proves the chip itself is genuine and not a copy. This is the strongest document check available in a KYC flow, far stronger than anything done from a photo of the page. The rules are in &lt;strong&gt;ICAO Document 9303&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why read the chip at all?
&lt;/h2&gt;

&lt;p&gt;Photo-based checks (&lt;a href="https://aliahmed.si/blog/passport-mrz-check-digits/" rel="noopener noreferrer"&gt;OCR, MRZ check digits&lt;/a&gt; and visual authenticity) can be fooled by a good forgery, because they only look at pixels. The chip's contents are signed with the issuing state's private key. If the signature verifies, the name, dates and photo are exactly what the government issued. A forger can print a convincing page but can't produce a valid signature.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: unlock the chip with the MRZ
&lt;/h2&gt;

&lt;p&gt;The chip won't talk to just any reader. Access control prevents someone from skimming a passport inside a bag:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;BAC (Basic Access Control):&lt;/strong&gt; the reader derives session keys from three MRZ fields: &lt;strong&gt;document number, date of birth and expiry date&lt;/strong&gt;, each with its check digit. So the app first reads the MRZ with the camera, which is one reason &lt;a href="https://aliahmed.si/blog/passport-mrz-check-digits/" rel="noopener noreferrer"&gt;MRZ accuracy&lt;/a&gt; matters.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;PACE (Password Authenticated Connection Establishment):&lt;/strong&gt; the modern replacement, using the same MRZ data (or a printed card access number) but a stronger key agreement. EU documents require it, and most new passports support it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Step 2: read the data groups
&lt;/h2&gt;

&lt;p&gt;The chip stores data in &lt;strong&gt;data groups (DGs)&lt;/strong&gt;. The two KYC needs most:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Data group&lt;/th&gt;
&lt;th&gt;Contents&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;DG1&lt;/td&gt;
&lt;td&gt;The MRZ data&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DG2&lt;/td&gt;
&lt;td&gt;The facial image&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DG14&lt;/td&gt;
&lt;td&gt;Keys for chip authentication&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DG15&lt;/td&gt;
&lt;td&gt;Public key for active authentication&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SOD&lt;/td&gt;
&lt;td&gt;Hashes of every DG, signed by the state&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Step 3: passive authentication
&lt;/h2&gt;

&lt;p&gt;Passive authentication proves the data is &lt;strong&gt;genuine and unaltered&lt;/strong&gt;:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Hash each data group read, and compare with the hashes in the &lt;strong&gt;Document Security Object (SOD)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Verify the SOD's signature with the &lt;strong&gt;Document Signer Certificate&lt;/strong&gt; stored alongside it.&lt;/li&gt;
&lt;li&gt;Verify that certificate against the issuing country's &lt;strong&gt;CSCA&lt;/strong&gt; (Country Signing Certificate Authority) certificate, from a trusted list such as the &lt;strong&gt;ICAO PKD&lt;/strong&gt; or national master lists.&lt;/li&gt;
&lt;li&gt;Check revocation lists.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If all of that passes, DG1 and DG2 are authentic. The DG2 photo is then a high-quality reference for &lt;a href="https://aliahmed.si/blog/face-match-threshold-far-frr/" rel="noopener noreferrer"&gt;face matching&lt;/a&gt;, much better than a phone photo of the printed portrait.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: chip or active authentication
&lt;/h2&gt;

&lt;p&gt;Passive authentication proves the &lt;strong&gt;data&lt;/strong&gt; is genuine, but not that this &lt;strong&gt;chip&lt;/strong&gt; is the original: signed data can be copied onto another chip. So:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Active authentication (AA):&lt;/strong&gt; the chip signs a random challenge with a private key that can't be read out, matching the public key in DG15.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Chip authentication (CA):&lt;/strong&gt; a key agreement using DG14 that also sets up stronger session encryption.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Either one proves the chip holds a private key that never leaves it, so a clone fails.&lt;/p&gt;

&lt;h2&gt;
  
  
  What are the practical limits?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Device support:&lt;/strong&gt; NFC is standard on iPhones and most Android phones, but some users' phones lack it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;User experience:&lt;/strong&gt; users must hold the passport against the right spot on the phone for several seconds. Clear guidance matters as much as the cryptography.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Certificate coverage:&lt;/strong&gt; you need trusted CSCA certificates for every country you accept, and not every country publishes to the ICAO PKD.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;It proves the document, not the person.&lt;/strong&gt; You still need a &lt;a href="https://aliahmed.si/blog/face-match-threshold-far-frr/" rel="noopener noreferrer"&gt;face match&lt;/a&gt; and &lt;a href="https://aliahmed.si/blog/face-liveness-detection-explained/" rel="noopener noreferrer"&gt;liveness&lt;/a&gt; to prove the holder is the person on the chip.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Where it fits
&lt;/h2&gt;

&lt;p&gt;In &lt;a href="https://aliahmed.si/blog/how-kyc-identity-verification-works/" rel="noopener noreferrer"&gt;the KYC pipeline&lt;/a&gt;, NFC reading replaces most of the visual document checks when it's available, and photo-based checks with &lt;a href="https://aliahmed.si/blog/ocr-non-latin-scripts-arabic-urdu/" rel="noopener noreferrer"&gt;OCR for many scripts&lt;/a&gt; remain the fallback. More about my identity verification work at ShuftiPro is on the &lt;a href="https://aliahmed.si/" rel="noopener noreferrer"&gt;home page&lt;/a&gt;.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/nfc-epassport-chip-verification/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>kyc</category>
      <category>security</category>
      <category>identityverification</category>
      <category>mobile</category>
    </item>
    <item>
      <title>Getting reliable JSON from LLMs: schemas, validation, retries and a deterministic fallback</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:29:18 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/getting-reliable-json-from-llms-schemas-validation-retries-and-a-deterministic-fallback-1lhi</link>
      <guid>https://dev.to/quantumbreakz/getting-reliable-json-from-llms-schemas-validation-retries-and-a-deterministic-fallback-1lhi</guid>
      <description>&lt;p&gt;To get reliable structured data from an LLM, use &lt;strong&gt;four layers&lt;/strong&gt;: ask for a &lt;strong&gt;JSON Schema&lt;/strong&gt; (and use the provider's structured-output or tool-calling mode where available), &lt;strong&gt;validate the result in code&lt;/strong&gt; with a schema library such as Pydantic, &lt;strong&gt;retry once with the validation error&lt;/strong&gt; included in the prompt, and fall back to a &lt;strong&gt;deterministic default&lt;/strong&gt; if it still fails. Schema-constrained decoding fixes the shape, but not the &lt;strong&gt;content&lt;/strong&gt;: a perfectly valid object can still contain an invented number or a leftover &lt;code&gt;{name}&lt;/code&gt; placeholder, so business rules must be checked in code too.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why isn't "respond in JSON" enough?
&lt;/h2&gt;

&lt;p&gt;In plain prompting, models regularly:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;wrap the JSON in Markdown code fences or add a sentence before it,&lt;/li&gt;
&lt;li&gt;leave out required fields, or add fields you didn't ask for,&lt;/li&gt;
&lt;li&gt;return a number as a string, or a list where you wanted one item,&lt;/li&gt;
&lt;li&gt;produce valid JSON with &lt;strong&gt;wrong content&lt;/strong&gt;: a placeholder token, an empty string, a value outside the allowed set.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Any of these, passed to the next step of a pipeline, becomes a bug far from its cause.&lt;/p&gt;

&lt;h2&gt;
  
  
  Layer 1: give the model a schema
&lt;/h2&gt;

&lt;p&gt;Most major APIs now support structured outputs, either through a JSON Schema response format or through tool calling, where the tool's input schema defines the shape. Where the provider &lt;strong&gt;constrains decoding&lt;/strong&gt; to the schema, malformed JSON mostly disappears. Use it when you can. But keep layer 2 anyway: you may switch providers or models, use a gateway such as LiteLLM, or hit a model that doesn't support strict mode.&lt;/p&gt;

&lt;h2&gt;
  
  
  Layer 2: validate in code
&lt;/h2&gt;

&lt;p&gt;Define the shape once, and validate every response against it:&lt;br&gt;
&lt;/p&gt;

&lt;p&gt;```python title="schema.py"&lt;br&gt;
from typing import Literal&lt;br&gt;
from pydantic import BaseModel, Field, ValidationError&lt;/p&gt;

&lt;p&gt;class LeadSummary(BaseModel):&lt;br&gt;
    company: str = Field(min_length=1, max_length=200)&lt;br&gt;
    industry: Literal["saas", "ecommerce", "services", "other"]&lt;br&gt;
    employees: int | None = Field(default=None, ge=1)&lt;br&gt;
    summary: str = Field(min_length=20, max_length=600)&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


`LeadSummary.model_validate_json(text)` either returns a typed object or raises a `ValidationError` that says exactly which field failed and why. Limits such as lengths and allowed values belong in the schema, not in a comment in the prompt.

## Layer 3: retry with the error

When validation fails, send the error back. Models are good at fixing a mistake they're shown:



```python title="extract.py"
def extract(llm, prompt: str, attempts: int = 2) -&amp;gt; LeadSummary | None:
    message = prompt
    for _ in range(attempts):
        text = llm(message)
        try:
            return LeadSummary.model_validate_json(strip_fences(text))
        except ValidationError as error:
            message = f"{prompt}\n\nYour last answer was invalid:\n{error}\nReturn corrected JSON only."
    return None

def strip_fences(text: str) -&amp;gt; str:
    start, end = text.find("{"), text.rfind("}")
    return text[start : end + 1] if start &amp;gt;= 0 and end &amp;gt; start else text
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One retry fixes most failures. More than two attempts usually means the prompt or schema is wrong, and repeated retries only add cost and latency.&lt;/p&gt;

&lt;h2&gt;
  
  
  Layer 4: check the content, then fall back
&lt;/h2&gt;

&lt;p&gt;Schema-valid isn't the same as correct. Add checks that only your domain knows:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No unfilled placeholders&lt;/strong&gt; such as &lt;code&gt;{first_name}&lt;/code&gt; or &lt;code&gt;[COMPANY]&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Numbers must come from the source.&lt;/strong&gt; If the input never mentioned 500 employees, the output can't either; see &lt;a href="https://aliahmed.si/blog/stop-llm-inventing-numbers/" rel="noopener noreferrer"&gt;how I stop an LLM from inventing numbers&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cross-field rules,&lt;/strong&gt; for example an end date after a start date.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If the output still fails, return a &lt;strong&gt;deterministic fallback&lt;/strong&gt;: a template, a "needs review" status, or &lt;code&gt;None&lt;/code&gt; that the caller handles. In &lt;a href="https://aliahmed.si/blog/prompt-injection-scraped-data-llm-outreach/" rel="noopener noreferrer"&gt;LeadForge&lt;/a&gt;, an email that fails validation twice is replaced by an industry template filled in by code. A worse but predictable result beats an unpredictable one.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to log
&lt;/h2&gt;

&lt;p&gt;For every call: the model, the prompt version, whether it validated first time, the validation error if not, and whether the fallback was used. The &lt;strong&gt;fallback rate&lt;/strong&gt; is the most useful health metric for an LLM feature. When it rises, a model update or a prompt change has broken something, often silently.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Layer&lt;/th&gt;
&lt;th&gt;Catches&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Schema in the request&lt;/td&gt;
&lt;td&gt;Malformed JSON, missing fields&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Validation in code&lt;/td&gt;
&lt;td&gt;Wrong types, out-of-range values&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Retry with the error&lt;/td&gt;
&lt;td&gt;Most one-off mistakes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Content checks&lt;/td&gt;
&lt;td&gt;Placeholders, invented numbers, rule violations&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Deterministic fallback&lt;/td&gt;
&lt;td&gt;Everything else, safely&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Related: &lt;a href="https://aliahmed.si/blog/rag-retrieval-evaluation-metrics/" rel="noopener noreferrer"&gt;evaluating RAG systems&lt;/a&gt;, &lt;a href="https://aliahmed.si/blog/reduce-llm-api-costs/" rel="noopener noreferrer"&gt;cutting LLM API costs&lt;/a&gt;, and &lt;a href="https://aliahmed.si/blog/agent-tool-error-messages/" rel="noopener noreferrer"&gt;writing tool errors AI agents can act on&lt;/a&gt;. More on the &lt;a href="https://aliahmed.si/projects/" rel="noopener noreferrer"&gt;projects page&lt;/a&gt;.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/llm-structured-output-validation/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>llms</category>
      <category>aiengineering</category>
      <category>python</category>
      <category>reliability</category>
    </item>
    <item>
      <title>Why our AI agent looped forever: a context-compression bug in an LLM gateway</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:26:06 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/why-our-ai-agent-looped-forever-a-context-compression-bug-in-an-llm-gateway-190m</link>
      <guid>https://dev.to/quantumbreakz/why-our-ai-agent-looped-forever-a-context-compression-bug-in-an-llm-gateway-190m</guid>
      <description>&lt;p&gt;An LLM gateway that compresses context has to remember one thing: &lt;strong&gt;content the agent just asked to retrieve must never be compressed again.&lt;/strong&gt; In LiteLLM's Headroom compression, it was. For clients that run their own tool loop, such as Claude Code, the retrieved text went back through compression, was swapped for the same placeholder, and the agent either looped on the same hash forever or answered without the content. My fix, &lt;a href="https://github.com/BerriAI/litellm/pull/38591" rel="noopener noreferrer"&gt;LiteLLM PR #38591&lt;/a&gt;, was merged in September 2026.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;This is the technical deep dive. The story of making the contribution is in &lt;a href="https://aliahmed.si/blog/first-litellm-contribution/" rel="noopener noreferrer"&gt;how I got my first PR merged into LiteLLM&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How does context compression in an LLM gateway work?
&lt;/h2&gt;

&lt;p&gt;Long tool results, such as a large JIRA issue, eat the context window. LiteLLM's Headroom guardrail compresses them. Large content in a request is replaced with a compact marker like &lt;code&gt;&amp;lt;&amp;lt;ccr:HASH,…&amp;gt;&amp;gt;&lt;/code&gt;, and the original is kept on the gateway side.&lt;/p&gt;

&lt;p&gt;When the model needs the full text, it calls a retrieval tool (&lt;code&gt;headroom_retrieve&lt;/code&gt;) with the hash and gets the original back. That's &lt;strong&gt;CCR: compress, cache, retrieve.&lt;/strong&gt; Done right, it keeps context small without losing anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  What went wrong?
&lt;/h2&gt;

&lt;p&gt;The bug only appeared for clients that &lt;strong&gt;run their own tool loop&lt;/strong&gt;, where each round trip is a fresh request through the proxy. Here's the failing flow with Claude Code pointed at the proxy (&lt;code&gt;ANTHROPIC_BASE_URL&lt;/code&gt;) and the retrieve tool exposed through LiteLLM's MCP gateway:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Claude Code fetches a large JIRA issue through an MCP tool. It's full and correct.&lt;/li&gt;
&lt;li&gt;The next &lt;code&gt;/v1/messages&lt;/code&gt; request carries that result, and the gateway compresses the body to &lt;code&gt;&amp;lt;&amp;lt;ccr:HASH,…&amp;gt;&amp;gt;&lt;/code&gt;. This part is correct.&lt;/li&gt;
&lt;li&gt;The model calls &lt;code&gt;headroom_retrieve&lt;/code&gt; and gets the original description back. Also correct.&lt;/li&gt;
&lt;li&gt;Claude Code sends the next &lt;code&gt;/v1/messages&lt;/code&gt; request carrying the &lt;strong&gt;retrieved&lt;/strong&gt; text, and the gateway compresses it again, &lt;strong&gt;to the same hash&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;The model sees the placeholder it just resolved, retrieves it again, and loops, or gives up and answers from an empty skeleton.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Retrieval was effectively a no-op. Nothing crashed and no error was logged. The agent just quietly got worse.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why was it hard to spot?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;It depended on who runs the loop.&lt;/strong&gt; When the gateway runs tools itself, it can keep track of what it already expanded. When the client runs the loop, every request looks new.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The output looked plausible.&lt;/strong&gt; An agent summarising a ticket without the description body still produces a summary.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tool names vary.&lt;/strong&gt; The retrieve tool appeared as &lt;code&gt;headroom_retrieve&lt;/code&gt; or, through the MCP gateway, as &lt;code&gt;mcp__&amp;lt;server&amp;gt;__headroom_retrieve&lt;/code&gt;. Long names can also be truncated by clients.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  How did the fix work?
&lt;/h2&gt;

&lt;p&gt;The fix holds &lt;strong&gt;retrieved tool-result rows back from the compression service&lt;/strong&gt; and leaves everything else alone:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It recognises results from both &lt;code&gt;headroom_retrieve&lt;/code&gt; and &lt;code&gt;mcp__&amp;lt;server&amp;gt;__headroom_retrieve&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;When a long tool name has been truncated, it falls back to matching the &lt;strong&gt;tool-call id&lt;/strong&gt; that links the result to its call.&lt;/li&gt;
&lt;li&gt;It reuses the gateway's &lt;strong&gt;existing protected-row path&lt;/strong&gt;, so the rest of the history still compresses normally. Nothing outside retrieved content is exempt.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;After the fix, the same Claude Code session returns a summary that includes the real description body, and the agent stops looping.&lt;/p&gt;

&lt;h2&gt;
  
  
  How did I prove it?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Reproduced it end to end: a Vertex AI Claude model behind the proxy, compression on, the retrieve tool exposed through MCP, and the before/after transcripts in the PR.&lt;/li&gt;
&lt;li&gt;Added tests for both tool-name forms and the truncated-name fallback.&lt;/li&gt;
&lt;li&gt;Ran the full LiteLLM CI (lint, type checks, the schema sync check and the unit suites) before asking for review.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What's the lesson for agent infrastructure?
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Any layer that rewrites model context must know which content is load-bearing.&lt;/strong&gt; Compression, summarisation, redaction and caching all rewrite context. If they can't tell "data the agent explicitly asked for" from "old history", they can undo the agent's own actions, and the symptom will look like a dumb model rather than a bug.&lt;/p&gt;

&lt;p&gt;I fixed a second, unrelated LiteLLM issue in &lt;a href="https://aliahmed.si/blog/llm-router-savings-deployment-id/" rel="noopener noreferrer"&gt;why cost dashboards lie when two deployments share a model name&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Context compression is one of several cost levers; the others are in &lt;a href="https://aliahmed.si/blog/reduce-llm-api-costs/" rel="noopener noreferrer"&gt;how to cut LLM API costs&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;What a gateway like LiteLLM is for, and when you need one: &lt;a href="https://aliahmed.si/blog/what-is-an-llm-gateway-litellm/" rel="noopener noreferrer"&gt;what an LLM gateway is&lt;/a&gt;.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/litellm-context-compression-bug/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>aiengineering</category>
      <category>llms</category>
      <category>opensource</category>
      <category>debugging</category>
    </item>
    <item>
      <title>KYC vs AML vs KYB vs CDD: what each means, and where AI fits in each</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:23:55 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/kyc-vs-aml-vs-kyb-vs-cdd-what-each-means-and-where-ai-fits-in-each-8jh</link>
      <guid>https://dev.to/quantumbreakz/kyc-vs-aml-vs-kyb-vs-cdd-what-each-means-and-where-ai-fits-in-each-8jh</guid>
      <description>&lt;p&gt;&lt;strong&gt;KYC (Know Your Customer)&lt;/strong&gt; checks that a customer is who they say they are. &lt;strong&gt;AML (Anti-Money Laundering)&lt;/strong&gt; is the wider legal programme a regulated business runs to stop its services being used for laundering money or financing terrorism; KYC is one part of it. &lt;strong&gt;KYB (Know Your Business)&lt;/strong&gt; is KYC for companies: verifying the business and the people who own and control it. &lt;strong&gt;CDD (Customer Due Diligence)&lt;/strong&gt; is the level of checking applied to a customer, and &lt;strong&gt;EDD (Enhanced Due Diligence)&lt;/strong&gt; is the deeper version for higher-risk customers. Founders building fintech, marketplaces or crypto products meet all five terms in the first compliance conversation, so here's what each means and where software and AI help.&lt;/p&gt;

&lt;h2&gt;
  
  
  KYC: who is this person?
&lt;/h2&gt;

&lt;p&gt;The onboarding check most people have done: photograph an ID, take a selfie. Behind it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Document verification:&lt;/strong&gt; is the ID genuine and readable? See &lt;a href="https://aliahmed.si/blog/how-kyc-identity-verification-works/" rel="noopener noreferrer"&gt;how KYC verification works&lt;/a&gt;, &lt;a href="https://aliahmed.si/blog/passport-mrz-check-digits/" rel="noopener noreferrer"&gt;MRZ check digits&lt;/a&gt; and &lt;a href="https://aliahmed.si/blog/nfc-epassport-chip-verification/" rel="noopener noreferrer"&gt;NFC chip reading&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Biometric verification:&lt;/strong&gt; does the selfie match the ID (&lt;a href="https://aliahmed.si/blog/face-match-threshold-far-frr/" rel="noopener noreferrer"&gt;face match thresholds&lt;/a&gt;), and is it a live person (&lt;a href="https://aliahmed.si/blog/face-liveness-detection-explained/" rel="noopener noreferrer"&gt;liveness detection&lt;/a&gt;)?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Data checks:&lt;/strong&gt; address, date of birth, sometimes a database lookup.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is where computer vision and OCR do most of the work, and where I spent my time at ShuftiPro.&lt;/p&gt;

&lt;h2&gt;
  
  
  AML: the whole programme
&lt;/h2&gt;

&lt;p&gt;AML covers everything a regulated business must do, including:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Screening&lt;/strong&gt; customers against sanctions lists, politically exposed person (PEP) lists and adverse media.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Transaction monitoring&lt;/strong&gt; for patterns such as structuring (splitting payments to stay under reporting limits) or money moving rapidly through an account.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reporting&lt;/strong&gt; suspicious activity to the financial intelligence unit.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Record-keeping, training and audits.&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;KYC tells you who the customer is; AML asks whether what they do makes sense for who they are.&lt;/p&gt;

&lt;h2&gt;
  
  
  KYB: verifying a business
&lt;/h2&gt;

&lt;p&gt;For business customers you verify the company (registration, status, address) and its &lt;strong&gt;ultimate beneficial owners (UBOs)&lt;/strong&gt;: the people who ultimately own or control it, typically above a 25% ownership threshold, although rules vary by country. Ownership chains through holding companies in several countries are what make KYB slow. Each UBO then goes through KYC.&lt;/p&gt;

&lt;h2&gt;
  
  
  CDD and EDD: how deep to go
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Simplified due diligence&lt;/strong&gt; for demonstrably low-risk cases, where the rules allow it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CDD:&lt;/strong&gt; the standard level: identify and verify the customer, understand the purpose of the relationship, and monitor it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;EDD:&lt;/strong&gt; for higher risk, such as PEPs, high-risk countries or unusual structures: source of funds and wealth, more documents and senior approval.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The level is set by a &lt;strong&gt;risk assessment&lt;/strong&gt;, so a good onboarding system adapts the checks to the customer instead of applying the maximum to everyone.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where does AI help?
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Area&lt;/th&gt;
&lt;th&gt;Useful AI&lt;/th&gt;
&lt;th&gt;Keep deterministic&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Document checks&lt;/td&gt;
&lt;td&gt;Computer vision, multilingual OCR&lt;/td&gt;
&lt;td&gt;MRZ check digits, chip signatures&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Biometrics&lt;/td&gt;
&lt;td&gt;Face matching, liveness models&lt;/td&gt;
&lt;td&gt;Thresholds and the review band&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Name screening&lt;/td&gt;
&lt;td&gt;Fuzzy and transliteration-aware matching&lt;/td&gt;
&lt;td&gt;The list data and the match decision log&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Adverse media&lt;/td&gt;
&lt;td&gt;LLMs to summarise and classify articles&lt;/td&gt;
&lt;td&gt;Sources cited for every finding&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Transaction monitoring&lt;/td&gt;
&lt;td&gt;Anomaly detection to find new patterns&lt;/td&gt;
&lt;td&gt;Rule-based scenarios regulators expect&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;KYB&lt;/td&gt;
&lt;td&gt;Extracting ownership from registry documents&lt;/td&gt;
&lt;td&gt;The UBO calculation itself&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Two rules carry across all of it. &lt;strong&gt;Every decision needs a reason a person can read&lt;/strong&gt;, because a regulator or a rejected customer will ask. And &lt;strong&gt;LLM output must be verified before it counts&lt;/strong&gt;: a summary that invents a fact about a customer is a compliance failure. That's the subject of &lt;a href="https://aliahmed.si/blog/stop-llm-inventing-numbers/" rel="noopener noreferrer"&gt;how I stop an LLM from inventing numbers&lt;/a&gt; and &lt;a href="https://aliahmed.si/blog/llm-structured-output-validation/" rel="noopener noreferrer"&gt;getting reliable structured output from LLMs&lt;/a&gt;, and audit trails are covered in &lt;a href="https://aliahmed.si/blog/ai-agent-audit-log-hash-chain/" rel="noopener noreferrer"&gt;tamper-evident logs for AI agents&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;KYC:&lt;/strong&gt; verify the person.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;KYB:&lt;/strong&gt; verify the business and its owners.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CDD / EDD:&lt;/strong&gt; how deeply to check, by risk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AML:&lt;/strong&gt; the full programme: screening, monitoring, reporting, and KYC inside it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I build AI for identity verification and financial intelligence; more on the &lt;a href="https://aliahmed.si/" rel="noopener noreferrer"&gt;home page&lt;/a&gt; and the &lt;a href="https://aliahmed.si/projects/" rel="noopener noreferrer"&gt;projects page&lt;/a&gt;.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/kyc-vs-aml-vs-kyb/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>kyc</category>
      <category>compliance</category>
      <category>aiengineering</category>
      <category>fintech</category>
    </item>
    <item>
      <title>How can deployment platforms afford free tiers?</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:20:44 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/how-can-deployment-platforms-afford-free-tiers-3dm9</link>
      <guid>https://dev.to/quantumbreakz/how-can-deployment-platforms-afford-free-tiers-3dm9</guid>
      <description>&lt;p&gt;Deployment platforms can offer generous free tiers because &lt;strong&gt;they stopped renting you a machine and started renting you execution&lt;/strong&gt;. A VPS reserves CPU and memory for you around the clock. A modern platform shares compute across thousands of workloads and allocates it only while your code is actually handling a request. Idle apps, which is most apps, cost almost nothing.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;This is part 1 of my Modern Deployments series.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What did deployment look like before?
&lt;/h2&gt;

&lt;p&gt;A few years ago, deploying a web app usually meant renting a VPS: 2 vCPUs, 4 GB of RAM, 80 GB of SSD. Then you configured Nginx, installed Node, set up PM2, sorted out SSL, opened firewall ports, and hoped nothing crashed at 3 AM.&lt;/p&gt;

&lt;p&gt;Today most developers push to GitHub, click Deploy, and get a production URL in under a minute. It almost feels like servers disappeared. They didn't. Someone is still paying for CPUs, RAM, storage, bandwidth and electricity.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why is a dedicated server wasteful?
&lt;/h2&gt;

&lt;p&gt;A VPS allocates infrastructure &lt;strong&gt;before&lt;/strong&gt; your application receives any traffic. Whether your app serves one request a day or a million, that virtual machine stays powered on and keeps its CPU and memory reserved for you.&lt;/p&gt;

&lt;p&gt;Most side projects, portfolios, hackathon apps and internal tools spend well over 99% of their lifetime idle. Provisioning them as if they were constantly busy is the waste.&lt;/p&gt;

&lt;h2&gt;
  
  
  What changed?
&lt;/h2&gt;

&lt;p&gt;Platforms stopped giving every application its own long-lived server. Instead they built systems that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Pack many workloads onto shared machines&lt;/strong&gt;, isolated by containers or micro-VMs&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Start compute on demand&lt;/strong&gt; when a request arrives&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reclaim it&lt;/strong&gt; once the app has been idle for a while&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cache static output at the edge&lt;/strong&gt;, so most page views never touch compute at all&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The economics changed because the allocation model changed. The free tier exists not because servers got cheap, but because &lt;strong&gt;an idle application no longer needs dedicated infrastructure&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's the catch?
&lt;/h2&gt;

&lt;p&gt;Renting execution has real costs, and they show up as behaviour you need to understand:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cold starts&lt;/strong&gt; when compute has to be created for a request. See &lt;a href="https://aliahmed.si/blog/serverless-cold-starts-explained/" rel="noopener noreferrer"&gt;what a serverless cold start really is&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Usage limits&lt;/strong&gt; on execution time, bandwidth and invocations, which is where the business model makes its money once you grow.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Less control&lt;/strong&gt; over the machine your code runs on. See &lt;a href="https://aliahmed.si/blog/servers-you-deploy-to/" rel="noopener noreferrer"&gt;why your app doesn't run on a server anymore&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The rest of the series
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;How free tiers work (this post)&lt;/li&gt;
&lt;li&gt;&lt;a href="https://aliahmed.si/blog/servers-you-deploy-to/" rel="noopener noreferrer"&gt;Your app runs on a desired state, not a server&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://aliahmed.si/blog/serverless-cold-starts-explained/" rel="noopener noreferrer"&gt;What is a serverless cold start?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://aliahmed.si/blog/what-gets-deployed-to-production/" rel="noopener noreferrer"&gt;What actually gets deployed to production?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://aliahmed.si/blog/where-do-environment-variables-go/" rel="noopener noreferrer"&gt;Where do your environment variables go?&lt;/a&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Once you see "serverless" as &lt;em&gt;renting execution instead of machines&lt;/em&gt;, the rest of the series starts to make sense.&lt;/p&gt;




&lt;p&gt;Written by Ali Ahmed · &lt;a href="https://aliahmed.si/blog/how-free-tiers-work/" rel="noopener noreferrer"&gt;Original article&lt;/a&gt;&lt;/p&gt;

</description>
      <category>moderndeployments</category>
      <category>serverless</category>
      <category>cloud</category>
    </item>
    <item>
      <title>Face match thresholds: how false accepts trade against false rejects in identity verification</title>
      <dc:creator>Ali Ahmed</dc:creator>
      <pubDate>Tue, 17 Nov 2026 16:18:22 +0000</pubDate>
      <link>https://dev.to/quantumbreakz/face-match-thresholds-how-false-accepts-trade-against-false-rejects-in-identity-verification-49l7</link>
      <guid>https://dev.to/quantumbreakz/face-match-thresholds-how-false-accepts-trade-against-false-rejects-in-identity-verification-49l7</guid>
      <description>&lt;p&gt;Face verification turns two face images into &lt;strong&gt;embeddings&lt;/strong&gt; (vectors) and compares them. If their similarity is above a &lt;strong&gt;threshold&lt;/strong&gt;, they're declared the same person. Every threshold makes two kinds of mistake: a &lt;strong&gt;false match&lt;/strong&gt; (an impostor accepted, measured as &lt;strong&gt;FMR&lt;/strong&gt;, also called FAR) and a &lt;strong&gt;false non-match&lt;/strong&gt; (the real person rejected, &lt;strong&gt;FNMR&lt;/strong&gt;, also called FRR). Raising the threshold lowers one and raises the other. The right threshold isn't a property of the model; it's a business decision, made from data, about which mistake costs more.&lt;/p&gt;

&lt;h2&gt;
  
  
  How does face verification work?
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Detect and align&lt;/strong&gt; the face in each image (the selfie and the document portrait), so eyes and mouth sit in standard positions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Embed&lt;/strong&gt; each face with a recognition model. Similar faces map to nearby vectors.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Compare&lt;/strong&gt;, usually with cosine similarity.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Decide&lt;/strong&gt; with a threshold.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This is &lt;strong&gt;1:1 verification&lt;/strong&gt; ("is this the claimed person?"). It's different from 1:N identification ("who is this, out of a million people?"), where errors scale with the size of the database. KYC is mostly 1:1, as one stage of the &lt;a href="https://aliahmed.si/blog/how-kyc-identity-verification-works/" rel="noopener noreferrer"&gt;verification pipeline&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What are FMR and FNMR?
&lt;/h2&gt;

&lt;p&gt;To measure them, you need labelled pairs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Genuine pairs:&lt;/strong&gt; two images of the same person. FNMR is the share of these scoring &lt;strong&gt;below&lt;/strong&gt; the threshold.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Impostor pairs:&lt;/strong&gt; two images of different people. FMR is the share of these scoring &lt;strong&gt;at or above&lt;/strong&gt; the threshold.
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;```python title="Error rates at a threshold"&lt;br&gt;
import numpy as np&lt;/p&gt;

&lt;p&gt;def error_rates(genuine: np.ndarray, impostor: np.ndarray, threshold: float):&lt;br&gt;
    fnmr = float(np.mean(genuine &amp;lt; threshold))&lt;br&gt;
    fmr = float(np.mean(impostor &amp;gt;= threshold))&lt;br&gt;
    return fmr, fnmr&lt;/p&gt;

&lt;p&gt;def threshold_for_fmr(impostor: np.ndarray, target_fmr: float) -&amp;gt; float:&lt;br&gt;
    return float(np.quantile(impostor, 1 - target_fmr))&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;


`threshold_for_fmr` picks the threshold that lets through the target share of impostors. You then read FNMR at that threshold, and that tells you how many real customers will fail.

## How do you choose the threshold?

Start from the mistake that's more expensive, and fix its rate:

* **Fix FMR first** when a false accept is costly: bank accounts, payments, account recovery. Typical targets are 1 in 10,000 down to 1 in 1,000,000 impostor attempts. Then check that FNMR is acceptable.
* Plot **FNMR against FMR** across thresholds (a DET curve) to see the trade-off. A better model moves the whole curve down, which is the only way to improve both at once.
* **Send the uncertain middle to a human** instead of forcing a yes or no. A band just under the threshold, reviewed manually, often costs less than either kind of error.

Be careful with small numbers. Estimating an FMR of one in a million needs **millions of impostor pairs**; with 10,000 pairs, "zero false matches" says very little.

## What moves the numbers in real KYC?

Benchmarks use clean photos. Real verification compares a **phone selfie** with a **small, printed, sometimes years-old document portrait**, and the error rates move with:

* **Image quality:** glare on the card, low light, motion blur, heavy compression.
* **Ageing:** a passport photo can be ten years old. The gap between genuine scores and impostor scores shrinks as the age difference grows.
* **Demographics:** NIST's evaluations (NIST IR 8280, 2019) found error rates for many algorithms varied by age, sex and race, with false match rates often differing by large factors between groups. **Measure FMR and FNMR per group** on your own data before choosing one global threshold.
* **Attacks:** a high similarity score from a printed photo is still a "match". That's why [liveness detection](https://aliahmed.si/blog/face-liveness-detection-explained/) runs alongside face matching.

## The checklist

* Pick the threshold **from your own genuine and impostor pairs**, not a vendor default.
* **Fix the costly error rate first**, then measure the other.
* Report rates **with sample sizes**, and per demographic group.
* Keep a **manual review band** for borderline scores.
* **Log scores, not just decisions**, so you can re-tune later without re-running every verification.

Related: [how passport MRZ check digits work](https://aliahmed.si/blog/passport-mrz-check-digits/), [OCR for Arabic, Urdu and other scripts](https://aliahmed.si/blog/ocr-non-latin-scripts-arabic-urdu/), and why a classifier's confident output still needs checking, from [my AI code detector post-mortem](https://aliahmed.si/blog/ai-code-detector-wrong-model/). More of my work is on the [home page](https://aliahmed.si/).

---

Written by Ali Ahmed · [Original article](https://aliahmed.si/blog/face-match-threshold-far-frr/)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>kyc</category>
      <category>computervision</category>
      <category>machinelearning</category>
      <category>python</category>
    </item>
  </channel>
</rss>
