DEV Community: Tae Kim

Building an investing knowledge graph, part 5: what LIVE actually means

Tae Kim — Wed, 02 Sep 2026 02:37:13 +0000

Building an investing knowledge graph, part 5: what LIVE actually means

Part 4 ended with the resolver working locally and me calling it live on Railway. That framing glossed over a gap. Something running locally and something running in production are different in ways that are obvious in retrospect and invisible until they bite you.

Here's what changed once I had a caller that wasn't me.

The first thing that breaks when someone else calls your API

The investing knowledge graph pipeline was the only caller for a long time. I knew its call patterns. I'd written both sides. The error budget was implicit — if something failed, I fixed it and kept going.

The first external caller changed that. When the API started returning errors for them, there was no "I'll fix it and retry." There was a broken integration on their end, and they had no context for diagnosing it.

The specific failure was unhelpful: a 500 with a generic error message on a request that had worked in my own testing. It took debugging from both ends to isolate the cause. The request was valid. The issue was that it hit the API during a window when the registry backend was mid-write, and the response came back internally inconsistent — some aliases resolved correctly, others returned stale data.

This wasn't a bug in the resolver logic. It was a backend durability problem I hadn't needed to care about when I was the only caller.

What the backend switch actually fixed

Locally, I used a file-based registry backend. The registry serializes to a JSON file, loaded into memory at startup and read during inference. Writes append and periodically flush to disk. Simple, fast, no dependencies.

The failure mode is obvious once you see it: during a write, if a read hits a partially flushed state, you get inconsistency. For a single-user pipeline running sequentially, this almost never surfaces. You'd have to be unlucky for a read to interleave with a write in a way that causes problems. Once there are concurrent callers, that probability stops being unlucky and becomes predictable.

The production backend is now PostgreSQL via Supabase. Writes go through transactions. Reads get a consistent snapshot. The alias table and the entity registry update atomically — either both change or neither does. A caller mid-request gets a consistent registry view regardless of what writes are in flight.

The switch is controlled by an environment variable (ER_REGISTRY_BACKEND). The application logic doesn't change, just the persistence layer. This separation made the transition straightforward: I could test both backends against the same code without restructuring anything.

The healthcheck

Railway handles deployments by routing traffic only after a service passes its healthcheck. The /health endpoint returns 200 when the API is ready to serve requests and the registry backend is reachable.

This matters in practice because startup isn't instantaneous. The service loads the model, initializes the Splink configuration, and verifies the backend connection before reporting healthy. Without an accurate healthcheck, Railway might route traffic to an instance that's running but not ready, which produces the kind of half-initialized state that generates confusing errors.

During a deploy, Railway keeps the old version serving traffic until the new one passes its healthcheck. No request reaches the new version before it's ready. This zero-downtime behavior is Railway's default — I didn't configure it specially — but it only works correctly if the healthcheck accurately reflects whether the service is ready.

The healthcheck has failed in two ways across different deployments: backend unreachable, and model initialization error. Both are things I'd rather catch at startup than discover mid-request. The healthcheck turned each into a failed deploy rather than a silent production problem.

Registry integrity at 47,853 entities

The registry currently holds 47,853 resolved entities and 47,883 aliases. Most entries are stable. Some are probably wrong.

Early in the project, when I was still calibrating the Splink threshold, the model was more permissive than it should have been. Some merges happened that were likely incorrect — entity strings that shared enough surface similarity to pass the threshold but were different companies. A few of those early-period decisions are still in the registry.

I know this in general. I don't know specifically which ones without re-running the full corpus against the current model. That re-run is on the list but hasn't happened yet.

What I have now is a review layer before new merges commit to the stable registry. Pairs that the model scores as probable matches but below a high-confidence threshold go into a pending state rather than being committed immediately. They get reviewed before joining the stable set. This doesn't fix the early errors already in the registry, but it stops new errors from accumulating at the same rate.

The production API serves only the stable registry. Pending pairs aren't exposed to external callers. If the model is uncertain, the caller gets a split decision. They can re-query later once additional evidence has accumulated and the pair has cleared review.

What "beta" actually means here

The last few posts in this series have mentioned "beta" without being specific. Here's what it actually implies:

The API is running. External callers are using it. I'm expanding access carefully rather than opening it wide.

It's not a prototype — the infrastructure decisions described above are real, the registry has nearly 48k entities built from actual article processing, and the resolver has been running reliably for months. It's also not a mature product with SLAs, a dedicated support tier, or uptime commitments beyond what Railway's infrastructure provides.

If you have an entity resolution problem — corporate name disambiguation, customer record deduplication, knowledge graph construction from heterogeneous sources — and want to try the service, there's a form at hannune.ai. I'll respond directly.

Next: the final part of this series wraps up the full arc and looks at where the same five-step pattern — DB limits, knowledge graph, ER bottleneck, direct implementation, cloud deployment — shows up outside investing graphs.

Built on Splink for probabilistic record linkage. Part 1 is here. Part 2 is here. Part 3 is here. Part 4 is here.

Building an investing knowledge graph, part 4: building the resolver

Tae Kim — Tue, 01 Sep 2026 02:37:00 +0000

Building an investing knowledge graph, part 4: building the resolver

Part 3 ended with a question I left open: how does the threshold actually get set, and what happens to the registry when new articles keep coming in? That's what this one covers.

The cost of getting it wrong in both directions

Before I touch thresholds, it's worth being specific about what failing in each direction costs.

A false merge combines two distinct entities into one node. In the Samsung SDI case from part 3, that would mean battery-supply disruptions show up when you traverse from the semiconductor business. The graph gives you an answer. The answer is wrong. You don't necessarily know it's wrong without already knowing the answer, which defeats the point.

A false split keeps the same entity as two separate nodes. "Samsung Electronics" and "the largest memory chipmaker in the world" stay disconnected. Edges pile up on each independently. A traversal from one doesn't reach the other. The graph gives you a partial answer. You lose coverage.

These aren't symmetric. A false merge injects noise that's hard to detect. A false split just means a disconnected node — you miss some connections, but you don't get fabricated ones.

So I set the threshold conservatively. Pairs below a high confidence mark stay split. Some real aliases end up never linking. That's the tradeoff I chose: lower recall, higher precision. For a graph I'm using to make judgments about which news matters, a confidently wrong edge is worse than a missing one.

What the endpoint does

The resolution service exposes a /v1/splink-pairs endpoint. It takes a list of candidate entity mention strings and returns a score for each pair, along with whether that pair crosses the registry threshold.

A minimal request looks like this:

POST /v1/splink-pairs
{
  "candidates": [
    "Samsung Electronics Co.",
    "the largest memory chipmaker in the world",
    "Samsung SDI",
    "TSMC"
  ]
}

The response returns a score matrix for pairs that the model considers worth evaluating — not every combination, just the ones above a blocking threshold that filters out obviously unrelated pairs before the full model runs.

{
  "pairs": [
    {
      "left": "Samsung Electronics Co.",
      "right": "the largest memory chipmaker in the world",
      "match_probability": 0.96,
      "decision": "match",
      "canonical_id": "company:samsung_electronics"
    },
    {
      "left": "Samsung Electronics Co.",
      "right": "Samsung SDI",
      "match_probability": 0.12,
      "decision": "split",
      "canonical_id": null
    }
  ]
}

The caller gets a decision (match or split) and the canonical entity ID when there's a match. If the match is against a known registry entry, the alias gets recorded. If it's a new pair the model hadn't seen before, that pair gets added to the evidence set for future model updates.

How the registry grows

When the investing knowledge graph pipeline processes a new article, it extracts entity mentions and runs them through the resolver. Three things can happen:

Known alias. The string matches an existing entry in the alias table. Lookup is fast, no model call needed. The mention gets written to the graph under the existing canonical ID.

Unknown mention, matches existing entity. The string isn't in the alias table, but the model scores it as a probable match against an existing entity. The alias gets added to the registry. Future articles that use the same string skip the model call.

Genuinely new entity. The model doesn't find a confident match against anything in the registry. A new canonical entry gets created. It starts small — one mention, no resolved aliases — and accumulates evidence as future articles mention the same company.

The registry currently has 47,853 resolved entities and 47,883 aliases. A lot of those started as single-mention nodes. Some have since merged as more articles confirmed the connection.

A few early decisions are probably still wrong. In the first batches I was more aggressive with merging, before I tightened the threshold. There are likely some nodes that should be split. I know this in general; I don't know specifically which ones without re-running the full corpus against the current model, which I haven't done.

What Splink is doing under the hood

Splink is the open source probabilistic record linkage library the model is built on. It uses DuckDB as the computation backend for the pair scoring, which handles the candidate generation and blocking step — filtering the space of possible pairs before running the full model.

For corporate entity mentions, the features that ended up mattering:

String similarity on the canonical name: catches abbreviations and common shorthand
Alias coverage: once a surface form is in the registry, future instances match by lookup rather than model score — this is how "the largest memory chipmaker in the world" eventually becomes reliable without its string similarity to "Samsung Electronics" ever improving
Token overlap on co-occurring named entities within the same article: an article about DRAM yields tends to mention different companies than an article about battery chemistry
Base rate weighting: a rare entity (one mention, no confirmed aliases) shouldn't get absorbed into a high-frequency entity just because they share a token

Splink learns the feature weights from labeled pairs. The training set was several hundred manually verified matches and non-matches from the first two thousand articles. Not large. But I had enough confirmed Samsung/Samsung SDI pairs to get the context features weighted correctly.

The point where this became a service

Somewhere around the third project where I had the same entity resolution problem — merging customer records from two systems, with the same fragmentation-versus-false-merge tradeoff — I stopped treating this as a per-project function and wrapped it into a standalone FastAPI service.

The knowledge graph pipeline now calls it over HTTP. Other things call it the same way. The registry is shared across callers. A match that one pipeline discovers helps every other caller that sees the same entity later.

That service is what I'm calling ER API. It runs on Railway, it's live, and the registry it's backing is the same one that's kept the investing knowledge graph from turning into a tangle of phantom nodes.

Next: what it takes to keep this running in production — the parts that only showed up after the first real caller wasn't me.

Built on Splink for probabilistic record linkage. Part 1 is here. Part 2 is here. Part 3 is here.

Building an investing knowledge graph, part 3: same company, seven names

Tae Kim — Mon, 31 Aug 2026 02:58:56 +0000

Building an investing knowledge graph, part 3: same company, seven names

I noticed a problem because a traversal that had been working stopped returning results. The ASML supply chain path I'd tested a week earlier came back empty. I spent a while thinking I'd broken the query. The query was fine. What had happened was that a new batch of articles from a different source used "the Dutch lithography equipment maker" and "ASML Holding NV" without ever using the string "ASML." Both landed in the graph as separate nodes. The traversal starting from the ASML node I'd been accumulating edges on couldn't reach them.

Seven names for one company

When I checked the Samsung Electronics entry in the registry, it had resolved seven distinct aliases to a single entity:

"samsung"
"samsung electronics"
"samsung electronics co."
"samsung foundry"
"samsung's"
"the korean company"
"the largest memory chipmaker in the world"

The foundry alias is a judgment call I went back on twice -- Samsung Foundry is technically a division with its own reporting, but in this corpus it was mostly used as shorthand for the parent in manufacturing capacity articles, so I kept it merged. The descriptive ones at the bottom are wire service shorthand. An article that opened with "Samsung Electronics" in the headline would use "the Korean company" three paragraphs later. Standard string matching returns near-zero similarity between "the largest memory chipmaker in the world" and "samsung electronics." They share no tokens.

Before I had a proper setup for this, I was using token overlap to match entity mentions. The threshold was loose enough to catch most of the alias variations above. It was also loose enough to merge Samsung SDI into Samsung Electronics.

Samsung SDI makes batteries. It is a separate publicly listed company. A lithium cell supply disruption hitting Samsung SDI's production line has nothing to do with Samsung Electronics' memory fab utilization. If those two ended up as the same node, events from one would show up in traversals that started from the other. I caught this specific case because I happened to be running a check on Samsung when a battery shortage article came through. Whether there are other pairs like this that I haven't caught, I genuinely don't know.

The descriptive ASML alias fails because it shares no tokens with the canonical name. Samsung SDI fails to separate because it shares the most salient token with an entity it should be distinct from. There's no threshold value that handles both. Loosening the threshold enough to catch the first kind of failure causes more of the second. These are actually different problems.

What Splink does with this

Splink treats entity resolution as a classification problem. For any candidate pair of mentions, the model computes a match probability using multiple features simultaneously and learns weights from a training set of confirmed pairs and non-matches.

For corporate entities in news text, the features I ended up using: canonical name similarity is the obvious starting point -- it handles "Samsung Electronics Co." but not "the largest memory chipmaker in the world." For descriptive references, alias coverage matters more: once a mention form has been confirmed as belonging to a given entity, future occurrences match by lookup. The ASML case gets better over time this way, even though the string overlap never improves.

For the Samsung SDI case, what made the difference was article-level context. An article that mentions DRAM yields and fab capacity almost certainly isn't about the battery subsidiary. Whether that context gets encoded as co-occurring industry terms or co-mentioned company names is a feature engineering choice; either way, it's information that string matching on the entity name alone can't see. I also added a base-rate feature after a few early merges looked wrong -- a rare entity with one mention shouldn't get absorbed into a high-frequency entity just because they share a token.

Splink learned weights across all of this from a training set of a few hundred manually verified pairs from the early corpus. Not large. But enough to get the Samsung SDI case right once I included the context features, which I hadn't initially.

The registry now has 47,853 resolved entities. That number has been climbing -- each new article either matches an existing entity or creates a new node that might get merged later as evidence accumulates. A few merges from early on are probably still wrong. Some entries I initially kept separate have since merged when more articles came in.

Still a threshold

Splink produces a probability, not a final answer. You still decide where to cut.

For this graph, I set that conservatively. A pair below a certain value stays split. Some valid aliases end up never linking to their entity -- nodes with few edges that mostly sit disconnected. I ran a check on a sample and most looked like genuine unknowns where there wasn't enough article evidence to confirm the connection. A few were probably real aliases I missed. That's the tradeoff: some fragmentation, in exchange for higher confidence that a merged entity represents one thing.

How that threshold gets set in practice, and what happens to the registry as new articles change the evidence picture, is part 4.

Built on Splink for probabilistic record linkage. Part 1 is here. Part 2 is here.

Building an investing knowledge graph, part 2: the query SQL can't answer

Tae Kim — Sat, 29 Aug 2026 02:52:23 +0000

Here is a Cypher query I can now run against the investing knowledge graph:

MATCH path = (source:Company {name: "ASML"})
             -[:SUPPLY_CUT|EXPORT_RESTRICTION|PRODUCTION_DELAY*1..4]->
             (downstream:Company)
RETURN 
  downstream.name AS company,
  length(path) AS hops,
  [r IN relationships(path) | r.article_id] AS evidence_articles,
  [r IN relationships(path) | r.event_date] AS event_dates
ORDER BY hops, event_dates[0]
LIMIT 30;

It returns paths. Not rows. Each result in the output is a chain from the source company to a downstream company, with the article ID and event date at every link. I can see whether NVIDIA appears in the results, how many steps away, and what sequence of articles establishes the connection. The depth bound is a parameter I tune at query time.

I didn't write this first. This is where I ended up after trying two other things that didn't work.

The search index

The article database sits in Elasticsearch, index called economic_news_articles_en, currently 9,414 documents. Starting with what I had:

GET economic_news_articles_en/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "text": "chip packaging export restriction" } },
        { "range": { "published_at": { "gte": "2023-01-01" } } }
      ]
    }
  },
  "_source": ["title", "published_at", "companies_mentioned"],
  "size": 15
}

Fourteen results. Some are on the specific restriction. A few match incidentally. The NVIDIA articles are nowhere in these results because NVIDIA isn't mentioned in any of them. This is not a deficiency in Elasticsearch. It returned exactly what it should. The question I wanted to ask is a different kind of question: it requires following edges between things, and the search index doesn't model edges. Relevance tuning would not have helped here.

The SQL attempt

I had generated a relational structure during the build process. An article_events table, each row an event type, a subject company, an affected company, and a date. The first query:

SELECT c.name, ae.event_type, ae.event_date
FROM article_events ae
JOIN companies c ON c.id = ae.object_id
WHERE ae.subject_id = (SELECT id FROM companies WHERE name = 'ASML')
  AND ae.event_type IN ('supply_cut', 'export_restriction', 'production_delay')
ORDER BY ae.event_date;

That gives first-hop results: companies directly named in events where ASML is the subject. Going one hop further:

WITH first_hop AS (
  SELECT ae.object_id AS affected_id, ae.event_date
  FROM article_events ae
  WHERE ae.subject_id = (SELECT id FROM companies WHERE name = 'ASML')
    AND ae.event_type IN ('supply_cut', 'export_restriction', 'production_delay')
)
SELECT DISTINCT c.name, ae2.event_type, ae2.event_date
FROM first_hop fh
JOIN article_events ae2 ON ae2.subject_id = fh.affected_id
JOIN companies c ON c.id = ae2.object_id
WHERE ae2.event_date >= fh.event_date
ORDER BY ae2.event_date;

I had to know upfront how many hops I was looking for. Two CTEs for two hops. A three-hop query needs three. If the connection I care about happens to be five steps deep, a three-hop query misses it quietly. Recursive CTEs handle variable depth but bring their own performance questions, and by that point I was building something that wasn't really a SQL use case anymore.

The schema itself was losing information. Events in articles don't always have clean subject-object pairs. An article might describe an event that involves three or four companies in ways I couldn't reduce to one row without discarding something.

Why the graph answered it

The Cypher query at the top of this post returned paths. The evidence traveled with the result. For each chain in the output I had the article IDs and the event dates at each intermediate link, not just the endpoint. The *1..4 depth parameter meant I could check four hops without rewriting anything.

The graph at this point holds 84,962 nodes and 275,293 relationships, built from those same 9,414 articles. On the Cypher query, response time is acceptable. When a path doesn't exist in the data, the query returns nothing, which is a correct answer. It means the graph doesn't have evidence for that connection, not that the connection doesn't exist in the world.

What the graph assumed

For the traversal to be useful, the same company needed to map to the same node regardless of how different articles referred to it.

That didn't happen automatically. Articles about ASML might say "ASML Holding NV" in a formal announcement, then a wire service shortens it to just "ASML," then a blog post never names the company and calls it "the Dutch lithography equipment maker" instead. If those land as three separate nodes in the graph, the traversal silently breaks. None of the three nodes accumulates enough edges to connect to anything interesting.

The flip side caused different problems. ASML Cymer is a subsidiary. It makes excimer laser light sources, not the lithography systems the parent is known for. Merging that name into the ASML parent node because the strings share characters would misattribute events. A supply disruption that hit only the subsidiary's production line would look, in the graph, like it hit ASML's main business.

Getting entity mentions to resolve correctly was the bottleneck I hadn't expected. That's what part 3 covers: how string similarity fails in both directions at once, and what probabilistic matching over multiple signals looks like in practice for 47,853 resolved entities.

Built on Splink for probabilistic record linkage. Part 1 is here.

When your calls can't leave your infrastructure: the compliance constraint that shapes the whole AI stack

Tae Kim — Fri, 28 Aug 2026 02:37:46 +0000

I wrote before about why I run speech-to-text locally—for me it started as a personal data hygiene decision. But when I've talked to other people about local AI pipelines, a different version of the same constraint comes up: not "I'd prefer the audio to stay local" but "the audio legally cannot leave our environment."

That's a different problem. A preference shapes your tooling choices. A compliance constraint shapes your architecture.

What the constraint actually looks like

The most common version I hear: enterprise customer agreements. A company builds a software tool that serves clients in regulated industries. Those clients have data handling requirements that specify data can only be processed in specific jurisdictions or on approved infrastructure. When that company wants to add AI-powered call analysis, every cloud transcription API is immediately out of scope—not because of policy preference, but because the customer contract requires it.

Attorney-client privilege is another version. Law firms and their clients have specific expectations about where privileged communications go. Using a third-party transcription service means the audio passes through systems outside that privileged relationship. Whether that technically breaks privilege is a legal question that most firms don't want to be the test case for.

Healthcare is the most formalized version: PHI (protected health information) that appears in patient calls needs to stay within HIPAA-compliant infrastructure. Some cloud providers offer BAAs (Business Associate Agreements) that extend that coverage, but plenty of healthcare organizations have decided the simplest answer is a processing pipeline that stays entirely on infrastructure they control.

The common thread: the constraint isn't about the transcription output (text) but about the audio itself, during processing. It's a data residency and custody question.

The architectural implication: it's not just transcription

Here's where this gets interesting from a systems perspective. If you accept the constraint that audio can't leave your infrastructure, and you're trying to build something useful from that audio—summaries, action items, entity extraction, follow-up flags—the constraint propagates through every step that touches the content.

You transcribe locally. But then what?

If you send the transcript to a cloud LLM for summarization, you've sent the content of the call off your infrastructure. The audio stayed local, but the information left. For most compliance constraints, this doesn't actually satisfy the requirement.

So the constraint ends up requiring:

Local transcription (Whisper or equivalent)
Local embeddings if you're doing semantic search across transcripts
Local LLM inference if you're doing summarization, extraction, or classification
Local storage, obviously

You end up needing a full local inference stack, not just a transcription component. The compliance requirement that started with "we can't use cloud STT" turns into "we need to run the whole pipeline on infrastructure we control."

This is the constraint I'm designing around

The setup I've been building runs Whisper for transcription, bge-m3 for embeddings, and Gemma for image/document processing—all on a single local server. I covered the resource management side of running three models on one box in an earlier post.

The "nothing leaves the box" constraint wasn't an afterthought. It was the first design decision, and it determined the whole stack. Every model choice, every pipeline component was evaluated against it: does this require sending data somewhere I don't control? If yes, it's out.

The tradeoffs are real. Local inference is slower than cloud APIs for equivalent hardware. You own the updates and maintenance. When a better Whisper model ships, you're updating your own deployment. The TCO calculation looks different depending on how much you're processing.

But for the use cases where the compliance constraint is genuine—where "it's in the customer agreement" or "it's a regulatory requirement"—those tradeoffs aren't optional. The architecture has to start from the constraint.

What I haven't built yet

The local inference stack is working. What I'm building toward is the layer that makes it actually useful: intake from multiple channels (calls, email, messages), process everything locally, surface what matters in a daily report.

The compliance constraint applies there too—not just transcription. Every step processes content that, in an NDA or regulated environment, can't transit external APIs. So the design for the report generation layer has to be local all the way through.

That's the hard part I'm working on now: not "can I transcribe calls locally" (yes, that's working), but "can I build the summarization and reporting layer locally at quality that's actually useful."

Other posts in this series: Running three AI models on one server | Why local STT over cloud API | Missed calls, missed revenue

Building an investing knowledge graph, part 1: from "does this news matter" to a graph I can query

Tae Kim — Mon, 24 Aug 2026 06:24:08 +0000

I keep hitting the same question when a headline crosses my feed about a company I'm holding, or thinking about holding: does this news actually reach the thing I care about? A chip export restriction hits some supplier nobody's heard of. Two weeks later a manufacturer I do care about moves, and the headline that started it never mentioned them once. I wanted to see that chain, the actual event-to-event path, and no sentiment score was going to show it to me.

So I tried the obvious thing first: search my article database for the company name and read whatever comes back.

Where the article database stopped helping

I have a database of financial news articles, 9,414 documents at the moment, each indexed with the companies, people, and events mentioned. "Show me articles about Samsung" works fine. That's what search is for. "Show me what happens downstream when a supplier gets sanctioned" falls apart immediately, because a table of articles has no concept of downstream. Each row is independent. Nothing connects article 4,201 about a supplier to article 6,830 about a company three steps down the chain (those two numbers are made up, but that's the shape of the problem). Unless I read every row myself and hold the connections in my head, which is what I was already doing, badly.

Filtering and sorting, a table does great. Walking a chain of "this caused that caused that" just isn't something a flat table does. The information might be sitting right there across a dozen rows. There's no native way to say "A affects B affects C." Tables were never built for that question.

Swapping rows for nodes and edges

A knowledge graph fixes this at the structural level. Instead of rows you have nodes (companies, people, events) and edges (supplies, owns, sanctioned, competes with), and once that structure exists, "what's downstream of this event" stops being a research project. You start at a node and walk outward.

That's what I've been building: articles get parsed for entities and events, those get written into a graph store, and a question like "what's connected to this supplier three hops out" becomes something the graph itself can answer. It currently holds 84,962 nodes and 275,293 relationships, all extracted from those same articles.

None of this is a novel idea. Graphs for multi-hop reasoning are well established. The part that hurt showed up one layer down, before any traversal logic even mattered.

The part that actually blocked me: the entities were wrong

A traversal is only as good as the nodes it's walking. My first batches of extracted articles produced a mess I didn't expect to be the hard part: the same company kept landing in the graph as several disconnected nodes, because the articles never call it the same thing twice.

Here's one company, exactly as the pipeline extracted it:

"Samsung"
"Samsung Electronics"
"Samsung Electronics Co."
"Samsung Foundry"
"Samsung's" (possessive, mid-sentence)
"the Korean company" (a lazy second reference)
"the largest memory chipmaker in the world" (same article, different lazy reference)

Seven surface forms, one real company. That last one is my favorite, I only caught it because a "company" node named an entire sentence looked absurd in the browser. Write each form into the graph as its own node and a query for everything connected to Samsung silently misses six of them. The causal chain I wanted literally can't form. The graph thinks it's looking at seven strangers.

And the failure runs the other way too, which took me longer to appreciate. Another article mentioned "삼성SDI": Samsung SDI, a real, separately listed subsidiary that makes batteries, not chips. Merge that into the Samsung Electronics node and the graph starts lying. SDI's battery contracts become evidence about a semiconductor business, and every chain through that node inherits the false premise.

So the bottleneck wasn't graph technology at all. It was entity resolution: given messy surface forms from unstructured text, decide which ones are the same real-world thing and which only look alike. Naive string matching fails both directions at once. "The largest memory chipmaker in the world" shares zero characters with "Samsung," and "Samsung SDI" is one string-edit away from a wrong merge. You need probabilistic matching over multiple signals, and a memory of what you've already resolved.

So I built it

I ended up building a standalone entity resolution service rather than a one-off cleanup function, mostly because this exact problem had already bitten me in every project that merges data from more than one source. It's built on Splink, the open source probabilistic record linkage library, wrapped in a FastAPI service. Feed it candidate mentions and it scores how likely two of them refer to the same entity, keeping a registry of resolved entities and known aliases so it gets better the more it sees. As of today the registry holds 47,853 resolved entities and 47,883 aliases. The Samsung example resolves correctly now: all seven mentions collapse into one node, Samsung SDI stays separate.

And it's live

The resolver runs as a deployed cloud service, sitting in front of the graph pipeline. Every incoming article gets its entity mess cleaned there before anything touches the investing knowledge graph.

If you're merging entities from more than one source, articles, KYC documents, supply chain records, customer files from separate systems, you will meet the Samsung problem eventually. That's what ER API is for. It's live and in beta. Next post: the resolver mechanics, how it decides "Samsung Foundry" and "the largest memory chipmaker in the world" are the same company without also swallowing Samsung SDI.

Missed calls, missed revenue: the local-first phone assistant I'm building

Tae Kim — Sat, 22 Aug 2026 02:34:47 +0000

Previous posts in this series covered the infrastructure: running three AI models sequentially on one server and why I run speech-to-text locally instead of calling a cloud API. This one is about the business problem I'm building toward.

The problem

I run a solo consulting practice. Client communication arrives over three channels: phone calls (including voicemails I check hours later), messaging apps, and email. None of these talk to each other.

After a busy stretch, I found myself doing a manual audit: scrolling back through four or five apps, trying to reconstruct what commitments I'd made, who I hadn't responded to, what questions were still open. It's time-consuming and I kept missing things.

The expensive miss is a phone call. Someone calls, leaves a voicemail, doesn't follow up in email. If I'm mid-task when the call comes in and don't process the voicemail until the next day, that lead has likely moved on. Industry data suggests small businesses miss 40–60% of incoming calls. I don't know exactly what my number is, but I've been that business.

The tools I evaluated each handle one channel. Transcription services for calls. Email summary tools for inbox. Nothing I found reads all three and produces a single output. The integration gap is where the problem actually lives.

What I'm building

The goal is a daily summary: here's who contacted you, here's what they wanted, here's what's still open, here are the likely follow-ups.

The infrastructure pieces are working individually. Whisper running locally handles call transcription—audio stays on my server, never goes to an external API. bge-m3 handles embeddings for semantic search. Gemma handles images and screenshots. All running sequentially on one box.

What I'm building is the layer on top: intake → process → report.

Two problems I've been figuring out this week:

1. What the output should actually contain

Before building the generator, I spent time designing the output format. A daily report isn't useful if it's just a list of transcripts. The items that actually need to surface:

Missed contacts: someone reached out and I haven't replied
Commitments: explicit statements I made that need follow-through ("I'll send that over by Friday")
Scheduling candidates: date/time mentions that might be pending appointments
Open questions: things they asked that I said I'd look into

The hard part wasn't picking the categories—it was making each one specific enough to actually implement. "Show me missed follow-ups" is ambiguous. "Flag any contact where they reached out and I have no outbound reply within 48 hours" is implementable. I went through the format item by item and rewrote vague categories as concrete conditions.

2. Linking the same person across channels

If someone calls Monday and emails Tuesday, those land in two separate systems with no shared ID. To produce a per-contact summary, I need to link them.

This is an entity resolution problem—the same class of problem I work on in my main client work, building ER pipelines for corporate data. The structure is identical: match records that don't share a unique identifier.

For comms data, the linking fields are usually phone number or email address. Complications:

People use multiple numbers (mobile, work, home)
Email aliases (john.smith@company.com and jsmith@company.com)
Display names don't match across apps

My current sketch: normalize phone numbers (strip formatting, expand country codes), match email with tolerance for common alias patterns, and handle name variants as a fallback signal rather than a primary match key. I haven't implemented this yet—this is still the design phase.

Where things stand

Working: Local transcription (Whisper), local embeddings (bge-m3), local VLM (Gemma). All running sequentially on one server. Audio, documents, and images stay local.

Building: The pipeline that takes call transcripts + email content + message content and produces the daily report. The output format is designed. The entity resolution approach is sketched. Nothing is integrated end to end.

The gap between "pieces working individually" and "system producing something I'd actually use daily" is what I'm working through now. Next concrete step: get the report generator to produce output I'd actually want to read, not just a proof of concept that technically runs.

Previously in this series: Running three AI models on one local server | Why local STT instead of a cloud API

Why I run speech-to-text locally instead of calling a cloud API

Tae Kim — Tue, 18 Aug 2026 02:34:34 +0000

Why I run speech-to-text locally instead of calling a cloud API

Yesterday I wrote about deploying gemma, bge-m3, and whisper on a single server without enough VRAM for all three. This post is about why whisper is one of those three.

The problem with cloud STT

When you call a cloud speech-to-text API—OpenAI's Whisper endpoint, Google Speech-to-Text, Amazon Transcribe—the audio travels to their servers. For most use cases that's fine. For mine it isn't.

The project I'm building transcribes work calls. Work calls contain client names, project specifics, sometimes pricing discussions. Sending that audio to a vendor's inference endpoint means it travels over the network and gets processed on hardware I don't control.

Running whisper locally means the audio file stays on the machine.

What the setup actually looks like

Local server: RTX 3060 (12 GB VRAM)
faster-whisper in int8 quantized mode
Audio files uploaded from a phone via a PWA, stored on the same machine, processed there

A transcription call looks like this:

from faster_whisper import WhisperModel

model = WhisperModel("medium", device="cuda", compute_type="int8")
segments, info = model.transcribe("recording.m4a", beam_size=5)
for segment in segments:
    print(f"[{segment.start:.1f}s] {segment.text}")

No API call. No auth header. No request log on a vendor's side. The audio file doesn't leave the machine.

The VRAM situation

The 12 GB card is shared with two other models. Whisper medium in int8 uses roughly 2.5–3 GB of VRAM when warm. That's workable as long as the models don't run at the same time.

The three models load sequentially in my pipeline: transcribe → embed → analyze. Whisper runs first, which means it has the most headroom before the other two have loaded. When gemma is handling a VLM task and the card is under pressure, whisper drops to CPU. Slower—30–60 seconds per minute of audio instead of 8–12—but it completes without crashing.

I didn't anticipate needing that fallback. It showed up during the first few real recordings when I hadn't fully worked out the load order yet.

What this doesn't solve

Local STT doesn't fix transcription accuracy automatically. Whisper medium is good but not perfect. Names, domain-specific terms, and cross-talk all degrade it. I haven't run it on enough actual work calls to have a reliable accuracy number. On clean audio, 90%+. On a speakerphone with background noise, noticeably worse.

It also doesn't solve the downstream pipeline. Right now I can transcribe a call and get a text file. The next piece—extracting structured information from that transcript, linking it to what came in over text and email—is what I'm working on now. That part isn't done.

Why not a hosted Whisper endpoint?

Hosted Whisper endpoints exist and some are cheap. If data residency doesn't matter, they're probably the right call: faster, no hardware to manage, no VRAM budgeting.

The asymmetry is the point. A slower local run that keeps audio in-house is worth the tradeoff when the content of the calls is the thing you're trying to protect. The cost of audio leaving is not symmetric with the cost of running it locally.

Where this fits

I'm building a phone assistant that collects what comes in over calls, texts, and emails—and turns it into something useful instead of leaving it scattered across three different apps. Local transcription is the piece that's working. Everything past the transcript is still being built.

If you've dealt with local Whisper deployment on shared GPU, curious what you found: do you bother with beam_size tuning, or does the accuracy delta not matter enough at inference time to be worth it?

Previous post in this series: Running three AI models on one local server when your VRAM doesn't cover all of them

Running three AI models on one local server when your VRAM doesn't cover all of them

Tae Kim — Mon, 17 Aug 2026 02:49:37 +0000

The first time I tried loading Whisper, bge-m3, and gemma at the same time on my local box, it OOMâ€™d immediately. Iâ€™d known this was going to happen, but I tried anyway to see where the ceiling actually was.

The machine is a workstation I already had, enough VRAM for any single model but not three simultaneously. I had some options: get more hardware, split across multiple machines, or figure out a sequential loading pattern. I went with sequential because adding hardware or managing separate boxes felt like problems I didn't want to solve yet.

Sequential loading means: load whatever model you need, use it, unload before moving on. Nothing runs in parallel. This was fine for my use case because the workloads don't actually overlap in time â€” I'vm not running embedding lookups while transcribing a recording. The load time overhead adds a few seconds per task. For Whisper, that's ~4 seconds on top of ~90 seconds of transcription for a 30-minute call, which I don't notice in practice.

The bge-m3 decision took the most deliberation

I started with all-MiniLM-L6-v2 for embeddings. It's small and fast. The issue was Korean/English mixed documents. I'm processing meeting summaries and reference docs that switch between the two, and MiniLM's cross-lingual similarity scores were unreliable enough that I was getting wrong nearest-neighbor results. bge-m3 handles cross-lingual matching better. It costs more VRAM and I had to drop the batch size from the default to stabilize it, but the accuracy difference on my actual data was clear.

One thing I didn't fully account for: when bge-m3 and Whisper are both unloaded, the load time for whichever comes next varies. bge-m3 seems to be slower on first load than subsequent loads in the same session, probably something to do with model weights caching at the OS level. I haven't investigated this properly.

Whisper was the easy pick

medium.en. I ran large once to compare on a client call recording and the accuracy improvement wasn't worth the extra VRAM cost for my use case. Transcription time for a 30-minute recording is under two minutes with medium. That's fast enough that I'm not sitting watching it.

Gemma was a coin flip between a few candidates

I needed local image analysis for screenshots and scanned documents. I tested gemma, a couple of LLaVA variants, and MiniCPM-V on maybe 20 images from my actual use case. Counted how many each model described correctly. Gemma came out ahead on document-heavy images. That test was informal enough that a different set of images might have given different results, but I had to pick something.

One thing I noticed later: gemma's processing time varies a lot by input image resolution. Standard screenshots are fast. A high-res photo from a phone camera takes noticeably longer. I still haven't profiled exactly what's happening there. If you're planning around latency, test with the actual image sizes you'll be using, not benchmarks from smaller inputs.

Where things stand

The sequential loading setup is stable. Iâ€™ve been running it for a few weeks without the OOM errors from the first attempt. The models work individually and I have code that chains them in sequence for a given input set.

What Iâ€™m still building is the layer that makes use of this: a pipeline that processes call recordings, emails, and messages together and produces something more useful than three separate outputs. The individual pieces are working. Connecting them into a coherent pipeline is the current work.

Originally posted at hannune.ai

Blocking Key Design for Entity Resolution: Why Name Normalization Fails East Asian Corporate Data

Tae Kim — Wed, 12 Aug 2026 02:40:31 +0000

Entity resolution at scale starts with a problem most introductions skip: you can't compare every pair of records. A dataset with 500,000 entities has 125 billion possible pairs. At one millisecond per comparison, exhaustive comparison takes 1,400 days.

Blocking reduces this to a tractable set. You partition records into blocks using one or more keys, and only compare records that share a key. The constraint: your blocking decision determines which true matches you'll never see. A match that falls into different blocks is a miss by design.

Building the entity resolution layer for 2asy.ai — a corporate intelligence knowledge graph for East Asian markets — blocking was where the approach broke down first, in a way that was hard to diagnose.

The Initial Approach: Name Normalization

The first blocking key was a normalized company name: strip punctuation, lowercase, remove common suffixes.

import re

COMMON_SUFFIXES = {
    "co., ltd.", "co.,ltd.", "co. ltd.", "ltd.", "inc.",
    "corp.", "corporation", "co.", "llc", "plc", "gmbh",
    "주식회사", "(주)", "(사)", "유한회사",
    "株式会社", "有限会社", "合同会社",
}

def normalize_name(name: str) -> str:
    name = name.lower().strip()
    name = re.sub(r'[^ws]', '', name)
    name = re.sub(r's+', ' ', name)
    for suffix in sorted(COMMON_SUFFIXES, key=len, reverse=True):
        if name.endswith(suffix):
            name = name[:-len(suffix)].strip()
    return name

Two records sharing a normalized name go into the same block. Simple, fast.

Where It Failed

The failure mode was Korean and Japanese company names. A company registered as "Samsung Electronics Co., Ltd." in a Korean regulatory filing might appear as "Samsung Electronics" in a Japanese document and "삼성전자" in a Korean tax filing.

After normalization, those three forms produce different blocking keys:

normalize_name("Samsung Electronics Co., Ltd.")  # → "samsung electronics"
normalize_name("Samsung Electronics")             # → "samsung electronics"  ← same block
normalize_name("삼성전자")                          # → "삼성전자"  ← different block!

The Korean script name falls into a separate block. It's never compared. The correct merge never happens.

The underlying issue: a blocking key derived from a single attribute assumes that field is stable across sources. It isn't. In East Asian corporate data, the company name field is the least stable field you have. It varies by language, script, legal suffix convention, transliteration standard, and filing jurisdiction.

Composite Blocking with Multiple Key Strategies

The approach that worked was generating multiple candidate keys per record and forming a candidate pair if they share any key.

from typing import Iterator

def generate_blocking_keys(record: dict) -> Iterator[tuple[str, str]]:
    """Yields (key_type, key_value) pairs for a company record."""
    name = record.get("name", "")
    normalized = normalize_name(name)

    if normalized:
        yield ("name_norm", normalized)

    entity_type = record.get("entity_type", "")
    jurisdiction = record.get("jurisdiction_code", "")
    if entity_type and jurisdiction:
        yield ("type_jurisdiction", f"{entity_type}_{jurisdiction}")

    romanized = romanize(name)
    if romanized:
        phonetic = soundex(romanized)
        yield ("phonetic", phonetic)

    reg_num = record.get("registration_number", "")
    if reg_num:
        yield ("reg_prefix", reg_num[:6])


def build_candidate_pairs(records: list[dict]) -> set[tuple[int, int]]:
    """Return all candidate pairs sharing at least one blocking key."""
    key_to_records: dict[tuple, list[int]] = {}

    for idx, record in enumerate(records):
        for key_type, key_value in generate_blocking_keys(record):
            bucket = (key_type, key_value)
            key_to_records.setdefault(bucket, []).append(idx)

    candidates: set[tuple[int, int]] = set()
    for indices in key_to_records.values():
        for i in range(len(indices)):
            for j in range(i + 1, len(indices)):
                a, b = sorted([indices[i], indices[j]])
                candidates.add((a, b))

    return candidates

A pair of records is a candidate if they share any key. This creates overlap — some pairs get compared more than once — but it eliminates the category of missed merges caused by name form variation.

Measuring the Impact

Adding the phonetic key increased candidate pairs by about 40% while recovering merges we'd been systematically missing.

def evaluate_blocking(records, ground_truth_pairs, blocking_fn):
    candidates = blocking_fn(records)
    true_matches = set(ground_truth_pairs)
    found = candidates & true_matches
    missed = true_matches - candidates
    recall = len(found) / len(true_matches) if true_matches else 0.0
    reduction_ratio = 1.0 - len(candidates) / (len(records) * (len(records) - 1) / 2)
    return {
        "recall": recall,
        "candidate_pairs": len(candidates),
        "reduction_ratio": reduction_ratio,
        "missed_true_matches": len(missed),
    }

# Name-only blocking:   recall 0.73, candidate_pairs 42_000, missed 12_500
# Composite blocking:   recall 0.94, candidate_pairs 59_000, missed 2_800

In a corporate ownership graph, the cost of a false negative is high. Two nodes for the same company means ownership relationships assigned to one aren't visible when querying through the other. The silent failure is worse than the computational overhead.

Blocking Key Design Doesn't End at Ship

Blocking key design isn't a one-time decision. As source variety increases, keys that worked for the initial source set start missing merges from new sources.

The monitoring approach: sample records that weren't merged, check manually whether they should have been, and track false negative rate over time. When it spikes, it usually means a new source has naming conventions that none of the current keys handle.

def sample_unmerged_for_review(
    records: list[dict],
    merged_ids: set[int],
    sample_size: int = 50,
) -> list[dict]:
    unmerged = [r for i, r in enumerate(records) if i not in merged_ids]
    by_jurisdiction: dict[str, list[dict]] = {}
    for r in unmerged:
        jur = r.get("jurisdiction_code", "UNK")
        by_jurisdiction.setdefault(jur, []).append(r)

    flagged = []
    for jur, recs in by_jurisdiction.items():
        total_in_jur = sum(1 for r in records if r.get("jurisdiction_code") == jur)
        merge_rate = 1.0 - len(recs) / total_in_jur
        if merge_rate < 0.3 and total_in_jur > 20:
            flagged.extend(recs[:10])

    return flagged[:sample_size]

The goal isn't zero false negatives — it's knowing which false negative rate your blocking tradeoffs are producing, and catching when that rate degrades without explanation.

I work on entity resolution and corporate knowledge graphs at er-api.hannune.ai and 2asy.ai. If you're designing blocking keys for multilingual or multi-script data, curious what strategies you've tried.

The Retry Counter Said 7, The Config Said 3

Tae Kim — Tue, 11 Aug 2026 02:58:04 +0000

A LangGraph pipeline I was reviewing last year. Token costs were running over projection, slow runs were taking about 3x the normal time. Nothing was throwing errors.

When I added a retry counter per step and ran a few slow cases through, the resolve step showed a count of 7. The config had max_retries=3.

The discrepancy came from two retry layers that had been written independently and never asked to coordinate. The step-level nodes had retry logic. The orchestrator graph had a recovery edge that re-invoked subgraphs when steps returned an error state — and that edge was triggering twice on ambiguous failure modes before the orchestrator gave up. So the actual retry count for those steps was whatever the step-level config said, plus two orchestrator re-invocations.

Once I had the counter, I could see it. Before that, I was looking at prompt quality and model selection because that's what the symptoms looked like.

The instrumentation was pretty simple: a counter object passed to every node, checked before retrying, incremented after.

import time
from collections import defaultdict

class RetryBudget:
    def __init__(self, run_limit=12, step_limit=3):
        self.run_limit = run_limit
        self.step_limit = step_limit
        self._total = 0
        self._by_step = defaultdict(int)

    def can_retry(self, step: str) -> bool:
        if self._total >= self.run_limit:
            return False
        return self._by_step[step] < self.step_limit

    def charge(self, step: str) -> None:
        self._total += 1
        self._by_step[step] += 1

    def state(self) -> dict:
        return {"total": self._total, "by_step": dict(self._by_step)}

run_limit=12 is meant to account for two layers interacting. If each step allows 3 retries and you have two layers that can both fire, you need the ceiling to be higher than 3 or the orchestrator layer will consume the whole budget on its first re-invoke. The team was using 12 which was basically 3 steps times 4 (step + orchestrator with some slack). It's not a principled number.

The orchestrator's recovery edge went through the same budget object. That was the main structural change — before, the orchestrator had no visibility into how many retries the step level had already used, and the step level had no idea the orchestrator was going to re-invoke. Sharing one budget object between both makes the interaction visible.

After a week of production data, _by_step showed that the resolve step was consuming most of the budget on slow runs. Looking at those specific calls, the upstream API was returning 200 with malformed JSON on a particular input pattern, and the retries were doing nothing because the response was consistently broken for that input. We ended up fixing input normalization before the call, not the retry config. But we wouldn't have known which step to look at without seeing where the retries were going.

The unresolved part is what to do when the run-level budget hits zero mid-run.

This pipeline accumulates meaningful intermediate state by the time it gets to the downstream steps. Early steps do entity resolution, later ones do candidate ranking, and by step 6 or 7 there's significant useful work already done. Failing the whole run discards that. Returning whatever completed without noting that the run didn't finish is wrong because the consumer runs aggregations on top of multiple runs, and incomplete results fed into aggregation without flagging produce incorrect totals.

What we ended up shipping was a result object with a complete: bool field and a list of which steps didn't finish. The consumer is supposed to check it before using the data. In practice, the check is inconsistent. The complete: False flag gets read in some call sites and ignored in others. The next thing we're considering is making the result type force the consumer to handle the incomplete case explicitly rather than just exposing a bool that can be ignored. Either a tagged union or a checked exception type. We haven't gotten there yet and I'm not certain it would actually fix the consistency problem even if we did.

There's probably a version where this becomes a dead letter queue and a human reviews the incomplete runs. We haven't needed that yet.

Why F1 Is the Wrong Objective for Entity Resolution Thresholds in Corporate Data

Tae Kim — Thu, 06 Aug 2026 02:35:36 +0000

Most entity resolution pipelines are tuned to maximize F1. For corporate data, that is the wrong objective.

F1 treats false positives and false negatives as equally costly. In practice they are not even close.

A false negative in corporate ER means you have two records that refer to the same company — a redundancy you can fix later by rerunning the pipeline with updated parameters. A false positive means you merged two distinct companies into one entity. Every query that touches that node now returns blended facts from two unrelated organizations. The damage is structural and hard to detect because the merged entity looks valid. It has real attributes, real edges, real documents. Downstream systems have no signal that anything is wrong.

The Production Failure That Exposed This

I ran into this building er-api, an entity resolution layer for trade and corporate registry data. The initial Splink model was tuned on an F1-maximizing grid search. It performed well on held-out test pairs. In production, it merged a South Korean steel manufacturer with a logistics subsidiary that shared part of its name and a city. Both entities were real, both had filings, and the combined record passed every sanity check that had been written for individual entities.

The merged node looked completely valid to any downstream query:

Real attributes from both companies
Edges connecting to real counterparties
Trade documents that appeared to corroborate each other

The error was invisible at the F1 level. It only surfaced when a user noticed that a steel exporter appeared to also own a refrigerated warehouse network.

The fix was not a better model. It was a different threshold structure.

Three Changes That Changed the Behavior

1. Split the decision space into three bands

Instead of a single match/no-match threshold, use three bands:

from dataclasses import dataclass
from enum import Enum

class ERDecision(Enum):
    AUTO_MERGE = "auto_merge"
    REVIEW_QUEUE = "review_queue"
    AUTO_REJECT = "auto_reject"

@dataclass
class ThresholdBands:
    auto_merge_floor: float = 0.95   # above this: auto-merge
    review_floor: float = 0.70       # between review_floor and auto_merge_floor: review queue
    # below review_floor: auto-reject

def classify_match(probability: float, bands: ThresholdBands) -> ERDecision:
    if probability >= bands.auto_merge_floor:
        return ERDecision.AUTO_MERGE
    elif probability >= bands.review_floor:
        return ERDecision.REVIEW_QUEUE
    else:
        return ERDecision.AUTO_REJECT

The bands were calibrated on domain-specific false positive cost, not test set F1. The auto-merge zone is deliberately narrow because the penalty for being wrong there is high. Most entities that a naive F1-tuned model would auto-merge end up in the review queue under this structure.

In practice: on a corpus of East Asian corporate entities, moving from a single threshold (0.85) to banded thresholds (0.95/0.70) reduced false positive rate by 73% with a 12% increase in review queue volume. The review queue increase is recoverable. The false positive reduction is not available at all under single-threshold tuning.

2. Require multiple blocking key agreement for auto-merge

A high match probability on name similarity alone is insufficient for the auto-merge band. The model must also agree on at least one structural attribute:

@dataclass
class AutoMergeGuard:
    """
    Name-only high-confidence matches route to review instead of auto-merge.
    At least one structural attribute must agree before auto-merge triggers.
    """
    structural_keys = [
        "registration_number_prefix",  # first 6 chars of company reg number
        "jurisdiction_code",            # ISO country + subdivision
        "address_locality",             # city-level normalized
    ]

    def check(self, candidate_pair: dict) -> bool:
        name_sim = candidate_pair.get("name_similarity", 0)
        if name_sim < 0.95:
            return True  # not name-only

        structural_agreement = any(
            candidate_pair.get(f"{key}_match", False)
            for key in self.structural_keys
        )
        return structural_agreement

This guard catches the class of error where two subsidiaries of different parent companies share a similar trading name in the same industry. Name similarity alone can push the Splink score above 0.95. The structural key requirement stops the auto-merge.

3. Track false positive rate separately

F1 hides false positive rate behind a harmonic mean. Pull it out as a first-class metric:

from datetime import datetime

class ThresholdMonitor:
    def __init__(self, fp_rate_alert_threshold: float = 0.001):
        self.decisions = []
        self.confirmed_fp = 0
        self.auto_merge_count = 0
        self.fp_rate_alert_threshold = fp_rate_alert_threshold

    def record_decision(self, decision: ERDecision, probability: float):
        self.decisions.append({
            "decision": decision,
            "probability": probability,
            "timestamp": datetime.utcnow()
        })
        if decision == ERDecision.AUTO_MERGE:
            self.auto_merge_count += 1

    def record_confirmed_false_positive(self):
        self.confirmed_fp += 1

    @property
    def false_positive_rate(self) -> float:
        if self.auto_merge_count == 0:
            return 0.0
        return self.confirmed_fp / self.auto_merge_count

    def is_alert(self) -> bool:
        return self.false_positive_rate > self.fp_rate_alert_threshold

The alert threshold matters here. A threshold that stays at 0.1% false positive rate on a corpus of millions of entity pairs produces recoverable errors — the review team can handle the correction volume. At 1%, the review queue cannot keep up and bad merges accumulate faster than they can be fixed. The graph degrades quietly.

The Principle

Optimizing for F1 is the right starting point for model selection. It is the wrong objective for threshold deployment.

In domains where a wrong merge is harder to reverse than a missed merge — corporate registries, medical records, financial entity graphs — the calibration step needs its own cost model. The cost model inputs are:

Estimated effort to detect a bad merge (can be zero if you have no downstream audit)
Estimated effort to correct a bad merge (graph surgery, downstream cache invalidation, restatement)
Estimated throughput of the review queue (determines how much to shift from auto-merge to review)

F1 assumes a symmetric cost. Most production entity graphs don't have a symmetric cost. The threshold structure should reflect that.

I'm building er-api as an entity resolution layer for East Asian corporate registry and trade document data. The asymmetric cost model described here is live in production. If you're running ER pipelines on corporate data and hitting similar false positive issues, I'm happy to compare notes.