DEV Community: Tae Kim

Blocking Key Design for Entity Resolution: Why Name Normalization Fails East Asian Corporate Data

Tae Kim — Wed, 12 Aug 2026 02:40:31 +0000

Entity resolution at scale starts with a problem most introductions skip: you can't compare every pair of records. A dataset with 500,000 entities has 125 billion possible pairs. At one millisecond per comparison, exhaustive comparison takes 1,400 days.

Blocking reduces this to a tractable set. You partition records into blocks using one or more keys, and only compare records that share a key. The constraint: your blocking decision determines which true matches you'll never see. A match that falls into different blocks is a miss by design.

Building the entity resolution layer for 2asy.ai — a corporate intelligence knowledge graph for East Asian markets — blocking was where the approach broke down first, in a way that was hard to diagnose.

The Initial Approach: Name Normalization

The first blocking key was a normalized company name: strip punctuation, lowercase, remove common suffixes.

import re

COMMON_SUFFIXES = {
    "co., ltd.", "co.,ltd.", "co. ltd.", "ltd.", "inc.",
    "corp.", "corporation", "co.", "llc", "plc", "gmbh",
    "주식회사", "(주)", "(사)", "유한회사",
    "株式会社", "有限会社", "合同会社",
}

def normalize_name(name: str) -> str:
    name = name.lower().strip()
    name = re.sub(r'[^ws]', '', name)
    name = re.sub(r's+', ' ', name)
    for suffix in sorted(COMMON_SUFFIXES, key=len, reverse=True):
        if name.endswith(suffix):
            name = name[:-len(suffix)].strip()
    return name

Two records sharing a normalized name go into the same block. Simple, fast.

Where It Failed

The failure mode was Korean and Japanese company names. A company registered as "Samsung Electronics Co., Ltd." in a Korean regulatory filing might appear as "Samsung Electronics" in a Japanese document and "삼성전자" in a Korean tax filing.

After normalization, those three forms produce different blocking keys:

normalize_name("Samsung Electronics Co., Ltd.")  # → "samsung electronics"
normalize_name("Samsung Electronics")             # → "samsung electronics"  ← same block
normalize_name("삼성전자")                          # → "삼성전자"  ← different block!

The Korean script name falls into a separate block. It's never compared. The correct merge never happens.

The underlying issue: a blocking key derived from a single attribute assumes that field is stable across sources. It isn't. In East Asian corporate data, the company name field is the least stable field you have. It varies by language, script, legal suffix convention, transliteration standard, and filing jurisdiction.

Composite Blocking with Multiple Key Strategies

The approach that worked was generating multiple candidate keys per record and forming a candidate pair if they share any key.

from typing import Iterator

def generate_blocking_keys(record: dict) -> Iterator[tuple[str, str]]:
    """Yields (key_type, key_value) pairs for a company record."""
    name = record.get("name", "")
    normalized = normalize_name(name)

    if normalized:
        yield ("name_norm", normalized)

    entity_type = record.get("entity_type", "")
    jurisdiction = record.get("jurisdiction_code", "")
    if entity_type and jurisdiction:
        yield ("type_jurisdiction", f"{entity_type}_{jurisdiction}")

    romanized = romanize(name)
    if romanized:
        phonetic = soundex(romanized)
        yield ("phonetic", phonetic)

    reg_num = record.get("registration_number", "")
    if reg_num:
        yield ("reg_prefix", reg_num[:6])


def build_candidate_pairs(records: list[dict]) -> set[tuple[int, int]]:
    """Return all candidate pairs sharing at least one blocking key."""
    key_to_records: dict[tuple, list[int]] = {}

    for idx, record in enumerate(records):
        for key_type, key_value in generate_blocking_keys(record):
            bucket = (key_type, key_value)
            key_to_records.setdefault(bucket, []).append(idx)

    candidates: set[tuple[int, int]] = set()
    for indices in key_to_records.values():
        for i in range(len(indices)):
            for j in range(i + 1, len(indices)):
                a, b = sorted([indices[i], indices[j]])
                candidates.add((a, b))

    return candidates

A pair of records is a candidate if they share any key. This creates overlap — some pairs get compared more than once — but it eliminates the category of missed merges caused by name form variation.

Measuring the Impact

Adding the phonetic key increased candidate pairs by about 40% while recovering merges we'd been systematically missing.

def evaluate_blocking(records, ground_truth_pairs, blocking_fn):
    candidates = blocking_fn(records)
    true_matches = set(ground_truth_pairs)
    found = candidates & true_matches
    missed = true_matches - candidates
    recall = len(found) / len(true_matches) if true_matches else 0.0
    reduction_ratio = 1.0 - len(candidates) / (len(records) * (len(records) - 1) / 2)
    return {
        "recall": recall,
        "candidate_pairs": len(candidates),
        "reduction_ratio": reduction_ratio,
        "missed_true_matches": len(missed),
    }

# Name-only blocking:   recall 0.73, candidate_pairs 42_000, missed 12_500
# Composite blocking:   recall 0.94, candidate_pairs 59_000, missed 2_800

In a corporate ownership graph, the cost of a false negative is high. Two nodes for the same company means ownership relationships assigned to one aren't visible when querying through the other. The silent failure is worse than the computational overhead.

Blocking Key Design Doesn't End at Ship

Blocking key design isn't a one-time decision. As source variety increases, keys that worked for the initial source set start missing merges from new sources.

The monitoring approach: sample records that weren't merged, check manually whether they should have been, and track false negative rate over time. When it spikes, it usually means a new source has naming conventions that none of the current keys handle.

def sample_unmerged_for_review(
    records: list[dict],
    merged_ids: set[int],
    sample_size: int = 50,
) -> list[dict]:
    unmerged = [r for i, r in enumerate(records) if i not in merged_ids]
    by_jurisdiction: dict[str, list[dict]] = {}
    for r in unmerged:
        jur = r.get("jurisdiction_code", "UNK")
        by_jurisdiction.setdefault(jur, []).append(r)

    flagged = []
    for jur, recs in by_jurisdiction.items():
        total_in_jur = sum(1 for r in records if r.get("jurisdiction_code") == jur)
        merge_rate = 1.0 - len(recs) / total_in_jur
        if merge_rate < 0.3 and total_in_jur > 20:
            flagged.extend(recs[:10])

    return flagged[:sample_size]

The goal isn't zero false negatives — it's knowing which false negative rate your blocking tradeoffs are producing, and catching when that rate degrades without explanation.

I work on entity resolution and corporate knowledge graphs at er-api.hannune.ai and 2asy.ai. If you're designing blocking keys for multilingual or multi-script data, curious what strategies you've tried.

The Retry Counter Said 7, The Config Said 3

Tae Kim — Tue, 11 Aug 2026 02:58:04 +0000

A LangGraph pipeline I was reviewing last year. Token costs were running over projection, slow runs were taking about 3x the normal time. Nothing was throwing errors.

When I added a retry counter per step and ran a few slow cases through, the resolve step showed a count of 7. The config had max_retries=3.

The discrepancy came from two retry layers that had been written independently and never asked to coordinate. The step-level nodes had retry logic. The orchestrator graph had a recovery edge that re-invoked subgraphs when steps returned an error state — and that edge was triggering twice on ambiguous failure modes before the orchestrator gave up. So the actual retry count for those steps was whatever the step-level config said, plus two orchestrator re-invocations.

Once I had the counter, I could see it. Before that, I was looking at prompt quality and model selection because that's what the symptoms looked like.

The instrumentation was pretty simple: a counter object passed to every node, checked before retrying, incremented after.

import time
from collections import defaultdict

class RetryBudget:
    def __init__(self, run_limit=12, step_limit=3):
        self.run_limit = run_limit
        self.step_limit = step_limit
        self._total = 0
        self._by_step = defaultdict(int)

    def can_retry(self, step: str) -> bool:
        if self._total >= self.run_limit:
            return False
        return self._by_step[step] < self.step_limit

    def charge(self, step: str) -> None:
        self._total += 1
        self._by_step[step] += 1

    def state(self) -> dict:
        return {"total": self._total, "by_step": dict(self._by_step)}

run_limit=12 is meant to account for two layers interacting. If each step allows 3 retries and you have two layers that can both fire, you need the ceiling to be higher than 3 or the orchestrator layer will consume the whole budget on its first re-invoke. The team was using 12 which was basically 3 steps times 4 (step + orchestrator with some slack). It's not a principled number.

The orchestrator's recovery edge went through the same budget object. That was the main structural change — before, the orchestrator had no visibility into how many retries the step level had already used, and the step level had no idea the orchestrator was going to re-invoke. Sharing one budget object between both makes the interaction visible.

After a week of production data, _by_step showed that the resolve step was consuming most of the budget on slow runs. Looking at those specific calls, the upstream API was returning 200 with malformed JSON on a particular input pattern, and the retries were doing nothing because the response was consistently broken for that input. We ended up fixing input normalization before the call, not the retry config. But we wouldn't have known which step to look at without seeing where the retries were going.

The unresolved part is what to do when the run-level budget hits zero mid-run.

This pipeline accumulates meaningful intermediate state by the time it gets to the downstream steps. Early steps do entity resolution, later ones do candidate ranking, and by step 6 or 7 there's significant useful work already done. Failing the whole run discards that. Returning whatever completed without noting that the run didn't finish is wrong because the consumer runs aggregations on top of multiple runs, and incomplete results fed into aggregation without flagging produce incorrect totals.

What we ended up shipping was a result object with a complete: bool field and a list of which steps didn't finish. The consumer is supposed to check it before using the data. In practice, the check is inconsistent. The complete: False flag gets read in some call sites and ignored in others. The next thing we're considering is making the result type force the consumer to handle the incomplete case explicitly rather than just exposing a bool that can be ignored. Either a tagged union or a checked exception type. We haven't gotten there yet and I'm not certain it would actually fix the consistency problem even if we did.

There's probably a version where this becomes a dead letter queue and a human reviews the incomplete runs. We haven't needed that yet.

Why F1 Is the Wrong Objective for Entity Resolution Thresholds in Corporate Data

Tae Kim — Thu, 06 Aug 2026 02:35:36 +0000

Most entity resolution pipelines are tuned to maximize F1. For corporate data, that is the wrong objective.

F1 treats false positives and false negatives as equally costly. In practice they are not even close.

A false negative in corporate ER means you have two records that refer to the same company — a redundancy you can fix later by rerunning the pipeline with updated parameters. A false positive means you merged two distinct companies into one entity. Every query that touches that node now returns blended facts from two unrelated organizations. The damage is structural and hard to detect because the merged entity looks valid. It has real attributes, real edges, real documents. Downstream systems have no signal that anything is wrong.

The Production Failure That Exposed This

I ran into this building er-api, an entity resolution layer for trade and corporate registry data. The initial Splink model was tuned on an F1-maximizing grid search. It performed well on held-out test pairs. In production, it merged a South Korean steel manufacturer with a logistics subsidiary that shared part of its name and a city. Both entities were real, both had filings, and the combined record passed every sanity check that had been written for individual entities.

The merged node looked completely valid to any downstream query:

Real attributes from both companies
Edges connecting to real counterparties
Trade documents that appeared to corroborate each other

The error was invisible at the F1 level. It only surfaced when a user noticed that a steel exporter appeared to also own a refrigerated warehouse network.

The fix was not a better model. It was a different threshold structure.

Three Changes That Changed the Behavior

1. Split the decision space into three bands

Instead of a single match/no-match threshold, use three bands:

from dataclasses import dataclass
from enum import Enum

class ERDecision(Enum):
    AUTO_MERGE = "auto_merge"
    REVIEW_QUEUE = "review_queue"
    AUTO_REJECT = "auto_reject"

@dataclass
class ThresholdBands:
    auto_merge_floor: float = 0.95   # above this: auto-merge
    review_floor: float = 0.70       # between review_floor and auto_merge_floor: review queue
    # below review_floor: auto-reject

def classify_match(probability: float, bands: ThresholdBands) -> ERDecision:
    if probability >= bands.auto_merge_floor:
        return ERDecision.AUTO_MERGE
    elif probability >= bands.review_floor:
        return ERDecision.REVIEW_QUEUE
    else:
        return ERDecision.AUTO_REJECT

The bands were calibrated on domain-specific false positive cost, not test set F1. The auto-merge zone is deliberately narrow because the penalty for being wrong there is high. Most entities that a naive F1-tuned model would auto-merge end up in the review queue under this structure.

In practice: on a corpus of East Asian corporate entities, moving from a single threshold (0.85) to banded thresholds (0.95/0.70) reduced false positive rate by 73% with a 12% increase in review queue volume. The review queue increase is recoverable. The false positive reduction is not available at all under single-threshold tuning.

2. Require multiple blocking key agreement for auto-merge

A high match probability on name similarity alone is insufficient for the auto-merge band. The model must also agree on at least one structural attribute:

@dataclass
class AutoMergeGuard:
    """
    Name-only high-confidence matches route to review instead of auto-merge.
    At least one structural attribute must agree before auto-merge triggers.
    """
    structural_keys = [
        "registration_number_prefix",  # first 6 chars of company reg number
        "jurisdiction_code",            # ISO country + subdivision
        "address_locality",             # city-level normalized
    ]

    def check(self, candidate_pair: dict) -> bool:
        name_sim = candidate_pair.get("name_similarity", 0)
        if name_sim < 0.95:
            return True  # not name-only

        structural_agreement = any(
            candidate_pair.get(f"{key}_match", False)
            for key in self.structural_keys
        )
        return structural_agreement

This guard catches the class of error where two subsidiaries of different parent companies share a similar trading name in the same industry. Name similarity alone can push the Splink score above 0.95. The structural key requirement stops the auto-merge.

3. Track false positive rate separately

F1 hides false positive rate behind a harmonic mean. Pull it out as a first-class metric:

from datetime import datetime

class ThresholdMonitor:
    def __init__(self, fp_rate_alert_threshold: float = 0.001):
        self.decisions = []
        self.confirmed_fp = 0
        self.auto_merge_count = 0
        self.fp_rate_alert_threshold = fp_rate_alert_threshold

    def record_decision(self, decision: ERDecision, probability: float):
        self.decisions.append({
            "decision": decision,
            "probability": probability,
            "timestamp": datetime.utcnow()
        })
        if decision == ERDecision.AUTO_MERGE:
            self.auto_merge_count += 1

    def record_confirmed_false_positive(self):
        self.confirmed_fp += 1

    @property
    def false_positive_rate(self) -> float:
        if self.auto_merge_count == 0:
            return 0.0
        return self.confirmed_fp / self.auto_merge_count

    def is_alert(self) -> bool:
        return self.false_positive_rate > self.fp_rate_alert_threshold

The alert threshold matters here. A threshold that stays at 0.1% false positive rate on a corpus of millions of entity pairs produces recoverable errors — the review team can handle the correction volume. At 1%, the review queue cannot keep up and bad merges accumulate faster than they can be fixed. The graph degrades quietly.

The Principle

Optimizing for F1 is the right starting point for model selection. It is the wrong objective for threshold deployment.

In domains where a wrong merge is harder to reverse than a missed merge — corporate registries, medical records, financial entity graphs — the calibration step needs its own cost model. The cost model inputs are:

Estimated effort to detect a bad merge (can be zero if you have no downstream audit)
Estimated effort to correct a bad merge (graph surgery, downstream cache invalidation, restatement)
Estimated throughput of the review queue (determines how much to shift from auto-merge to review)

F1 assumes a symmetric cost. Most production entity graphs don't have a symmetric cost. The threshold structure should reflect that.

I'm building er-api as an entity resolution layer for East Asian corporate registry and trade document data. The asymmetric cost model described here is live in production. If you're running ER pipelines on corporate data and hitting similar false positive issues, I'm happy to compare notes.

Incremental Graph Updates for Corporate Knowledge Graphs: Three Problems Batch Pipelines Can't Solve

Tae Kim — Tue, 04 Aug 2026 02:38:14 +0000

Corporate knowledge graphs break in production the moment you treat them as a batch artifact.

I ran into this building 2asy.ai, a Graph-RAG system for East Asian corporate intelligence. The initial architecture was standard: ingest a document set, run NER and entity resolution across the batch, write the graph. Clean, deterministic, fast to prototype.

Then the first update cycle hit.

The Three Problems That Show Up in Production

Problem 1: Entity Resolution at the Boundary

The batch pipeline handled entity resolution using blocking keys across the entire document corpus. With incremental updates, you only have the incoming batch. The blocking keys no longer reach the existing graph nodes.

In practice this means: a new trade document arrives with "Samsung Elec. Co., Ltd." The existing graph has "Samsung Electronics Co., Ltd." as a canonical node with 847 existing edges. Without live-graph entity resolution, you insert a second node and start building a disconnected duplicate.

Here is the resolution-at-boundary pattern that works:

from dataclasses import dataclass
from typing import Optional
import numpy as np

@dataclass
class EntityCandidate:
    raw_name: str
    normalized_name: str
    language: str
    doc_id: str
    embedding: Optional[np.ndarray] = None

def resolve_against_live_graph(
    candidate: EntityCandidate,
    graph_client,  # neo4j.GraphDatabase or similar
    embed_fn,
    threshold: float = 0.85,
) -> Optional[str]:
    """
    Run entity resolution against existing graph nodes before inserting.
    Returns the existing node ID if a match is found, else None.
    """
    # Step 1: Blocking — retrieve candidate neighbors by name prefix
    blocking_query = """
    MATCH (e:Entity)
    WHERE e.blocking_key STARTS WITH $prefix
    RETURN e.id AS id, e.canonical_name AS name, e.name_embedding AS emb
    LIMIT 50
    """
    prefix = candidate.normalized_name[:4].upper()
    neighbors = graph_client.run(blocking_query, prefix=prefix).data()

    if not neighbors:
        return None

    # Step 2: Embedding similarity against blocked candidates
    if candidate.embedding is None:
        candidate.embedding = embed_fn(candidate.normalized_name)

    best_score = 0.0
    best_id = None
    for row in neighbors:
        if row["emb"] is None:
            continue
        existing_emb = np.array(row["emb"])
        score = float(np.dot(candidate.embedding, existing_emb) /
                      (np.linalg.norm(candidate.embedding) * np.linalg.norm(existing_emb) + 1e-9))
        if score > best_score:
            best_score = score
            best_id = row["id"]

    return best_id if best_score >= threshold else None

If resolve_against_live_graph returns a node ID, you merge onto that node. If it returns None, you insert as new. The full entity resolution run happens at the boundary, not after insertion.

Problem 2: Supersession Without Time-Bounded Facts

A company relocates its headquarters. The new filing contains the correct address. The old record is already in the graph. Naive append gives you both — which means every Graph-RAG query touching that entity gets conflicting facts. The LLM has to guess which one is current.

The fix is to model each property value as a time-bounded assertion rather than a mutable attribute:

from datetime import date
from neo4j import GraphDatabase

def upsert_fact_with_supersession(
    driver: GraphDatabase,
    entity_id: str,
    property_name: str,
    new_value: str,
    valid_from: date,
    source_doc_id: str,
) -> None:
    """
    Write a new fact, terminating any currently-open assertion for the same property.
    Old assertions are retained for audit; only valid_to is set.
    """
    with driver.session() as session:
        session.run("""
            // Terminate the previous open assertion for this property
            MATCH (e:Entity {id: $entity_id})-[:HAS_FACT]->(f:Fact)
            WHERE f.property_name = $property_name
              AND f.valid_to IS NULL
            SET f.valid_to = date($valid_from) - duration('P1D')
        """, entity_id=entity_id, property_name=property_name,
             valid_from=str(valid_from))

        session.run("""
            // Insert the new assertion with open valid_to
            MATCH (e:Entity {id: $entity_id})
            CREATE (e)-[:HAS_FACT]->(f:Fact {
                property_name: $property_name,
                value:          $new_value,
                valid_from:     date($valid_from),
                valid_to:       null,
                source_doc_id:  $source_doc_id
            })
        """, entity_id=entity_id, property_name=property_name,
             new_value=new_value, valid_from=str(valid_from),
             source_doc_id=source_doc_id)

Queries then specify a point-in-time context:

// Get headquarters address as of 2024-03-01
MATCH (e:Entity {id: $entity_id})-[:HAS_FACT]->(f:Fact)
WHERE f.property_name = 'headquarters_address'
  AND f.valid_from <= date('2024-03-01')
  AND (f.valid_to IS NULL OR f.valid_to >= date('2024-03-01'))
RETURN f.value AS address

This is the pattern that makes Graph-RAG as-of queries reliable. Without it, you are relying on insertion order, which is neither guaranteed nor queryable.

Problem 3: Cascade Propagation After Node Merge

When two entity nodes merge (because resolve_against_live_graph found a match), the incoming node's edges need to be transferred to the surviving node. If your update pipeline does not model this explicitly, you end up with a partially disconnected graph where relationships point to deprecated node IDs.

def merge_nodes_and_transfer_edges(
    driver: GraphDatabase,
    surviving_id: str,
    deprecated_id: str,
) -> int:
    """
    Transfer all edges from deprecated node to surviving node,
    then mark deprecated node as merged.
    Returns the number of edges transferred.
    """
    with driver.session() as session:
        # Transfer outgoing edges
        result = session.run("""
            MATCH (deprecated:Entity {id: $deprecated_id})-[r]->(target)
            WHERE NOT target.id = $surviving_id
            MATCH (surviving:Entity {id: $surviving_id})
            CALL apoc.refactor.from(r, surviving)
            YIELD input, output
            RETURN count(*) AS transferred
        """, deprecated_id=deprecated_id, surviving_id=surviving_id)
        outgoing = result.single()["transferred"]

        # Transfer incoming edges
        result = session.run("""
            MATCH (source)-[r]->(deprecated:Entity {id: $deprecated_id})
            WHERE NOT source.id = $surviving_id
            MATCH (surviving:Entity {id: $surviving_id})
            CALL apoc.refactor.to(r, surviving)
            YIELD input, output
            RETURN count(*) AS transferred
        """, deprecated_id=deprecated_id, surviving_id=surviving_id)
        incoming = result.single()["transferred"]

        # Mark deprecated node
        session.run("""
            MATCH (e:Entity {id: $deprecated_id})
            SET e.merged_into = $surviving_id,
                e.merged_at   = datetime()
        """, deprecated_id=deprecated_id, surviving_id=surviving_id)

    return outgoing + incoming

The merged_into property lets you build a forwarding index. Any query that arrives with a deprecated node ID can be redirected to the surviving node without a full graph scan.

Tracking Dirty Neighbors

After a merge, the direct neighbors of the deprecated node may have consistency issues — edges that now point through the merged node incorrectly, or facts that reference the deprecated entity in their source context.

The lightweight fix is a dirty-neighbor queue:

def flag_dirty_neighbors(
    driver: GraphDatabase,
    node_id: str,
    reason: str,
) -> list[str]:
    """
    Flag the direct neighbors of a node for consistency recheck
    in the next ingestion cycle.
    """
    with driver.session() as session:
        result = session.run("""
            MATCH (n:Entity {id: $node_id})-[*1]-(neighbor:Entity)
            WHERE NOT neighbor.id = $node_id
            SET neighbor.dirty = true,
                neighbor.dirty_reason = $reason,
                neighbor.dirty_at = datetime()
            RETURN collect(DISTINCT neighbor.id) AS neighbor_ids
        """, node_id=node_id, reason=reason)
        return result.single()["neighbor_ids"]

At the start of each ingestion cycle, you pull all dirty neighbors and run a consistency check before processing new documents. This handles cascade propagation without a full graph traversal on every update.

The Pattern Under the Pattern

Batch rebuild works in a prototype. Production knowledge graphs need an explicit update model from the start:

Entity resolution at the boundary: run resolution against the live graph before insertion, not just within the incoming batch
Time-bounded facts: model property values as assertions with validity windows, not mutable attributes
Tracked propagation: flag dirty neighbors on merge so cascades are handled in the next cycle, not discovered by users

Adding these after the fact means rebuilding the schema and reprocessing historical documents. The three problems above are not edge cases — they are the production steady state for any graph that receives continuous updates.

I work on entity resolution and agentic infrastructure at er-api.hannune.ai. The Graph-RAG system this is based on runs against East Asian corporate registries where transliteration variance and entity deduplication are the hard problems.

Context window growth is the silent failure mode in agentic pipelines

Tae Kim — Sun, 02 Aug 2026 02:42:35 +0000

Context window management is the invisible cost in every multi-step agent pipeline.

I see this pattern across production consulting work. An agent that works cleanly in testing — ten steps, coherent reasoning, correct output — starts degrading six weeks after ship. No errors. No crashes. Just drift. Outputs get shorter. Reasoning steps that used to show clear logic start reading like vague summaries. Users start noticing that the agent "feels slower."

The root cause is almost always the same. Context grows with every step and nobody measured it.

How context accumulates

A multi-step agent accumulates its full conversation history by default. Every tool call result gets appended. Every intermediate reasoning step gets stored. The conversation object passed to each model call grows linearly with the number of steps executed.

By step 8 or 9 in a ten-step pipeline, the model is receiving 90% of the context window on each call. Technically within limits, but attention spread thin across thousands of tokens of earlier reasoning that no longer affects the current step.

Here is what this looks like in a real pipeline. Suppose a research agent runs a search, extracts entities, resolves ambiguities, retrieves documents, ranks them, synthesizes a draft, and fact-checks it. That is seven steps. If each step produces an average of 800 tokens of tool output and the model includes its full reasoning trace for each step:

After step 1: ~1,200 tokens in context
After step 3: ~4,800 tokens
After step 5: ~9,600 tokens
After step 7: ~15,600 tokens

For a 16K context model, that last step is operating with almost nothing left for the actual synthesis task. For a 128K context model, the numbers look more comfortable until you realize you are paying for all of that context on every call, and the model's attention is being diluted across material that stopped being relevant three steps ago.

Why testing does not catch this

Single-query testing never reveals context saturation. A developer runs the pipeline ten times against a test set, sees correct outputs, ships it. What they tested was ten fresh conversation threads, each starting at zero tokens.

Production load looks nothing like that. Users run sequential tasks. A pipeline that runs five requests across the same session accumulates context from all five runs if the session history is not explicitly bounded. An orchestration layer that retries failed steps passes the failure reasoning back into context. Long-running agents that call external APIs and wait for results accumulate tool logs while they wait.

Context growth under real query volume is not a scaling problem in the usual sense. It is a state management problem. The pipeline has no concept of which historical context is still relevant to the current step.

Three things that work in production

Allocate a fixed token budget per step, not per conversation.

If step 3 needs the output of step 1, pass the structured result only — not step 1's full chain-of-thought. A search step that returns ten results does not need to carry its internal ranking rationale forward into the synthesis step. What the synthesis step needs is the ranked list. Truncate aggressively at stage boundaries.

This requires a deliberate design decision about what each step's output contract is. If the output is unstructured text, everything gets passed forward by default. If the output is a typed schema, only the schema fields travel between steps. The type boundary is what enforces the token budget.

Summarize and compact before handoff between pipeline stages.

The detailed reasoning from a retrieval step does not need to be present when the synthesis step runs. A structured summary does. Before passing control from one stage to the next, run a compaction step that takes the full output and produces a fixed-size digest. The digest carries the key facts forward. The full reasoning trace stays available for debugging but does not enter the production call chain.

The compaction step is itself a model call, which costs something. It is almost always worth it. A 3,000-token reasoning trace compacted to a 400-token structured summary saves 2,600 tokens on every subsequent step in the pipeline. On a ten-step pipeline running thousands of queries, the compaction cost is negligible against the savings.

Set a hard context limit per step and fail loudly when exceeded.

Silent context overflow produces subtly wrong outputs that reach users. A model operating with 2% of its context window available for the actual task will not return an error. It will return a plausible-sounding but degraded answer that passes surface-level quality checks. The failure mode is invisible unless you are actively measuring context size per step.

Adding a hard limit is straightforward: measure the token count before each model call and raise an exception if it exceeds a threshold. The threshold should be well below the model's maximum, not a fraction of the maximum. If the step budget is 2,000 tokens and the accumulated context is 8,000, the pipeline is not "within limits" — it is already broken.

Failing loudly means the engineering team sees the problem. Silent degradation means users see it first.

What instrumentation you actually need

Three metrics catch context growth problems before they affect users:

Tokens-in per step: how many tokens the model receives at each pipeline step. Log this per step, not as a conversation total.
Context utilization rate: tokens-in divided by the model's context limit. Alert when this exceeds 70% for any step in a production pipeline.
Step-over-step growth rate: how much context is added between consecutive steps. A step that adds 5,000 tokens is probably passing its full reasoning trace forward.

These are not difficult to collect. A wrapper around the model call captures the prompt token count before sending. The difficulty is that most orchestration frameworks do not expose this by default. You have to add the instrumentation yourself.

The deeper design issue

The reason context growth catches teams off guard is that it does not look like a bug. The pipeline architecture that worked in a stateless test environment is also the architecture that fails under stateful production load. Nothing changed between test and production except the accumulated state.

The fix is to treat context as a resource with a budget, the same way you treat memory or API calls. Each step has an input budget and an output budget. The input budget caps what the step can receive. The output budget caps what the step passes forward. The pipeline enforces both.

An agentic pipeline that has no explicit context budget is running without a key operational constraint. It will work until it does not, and the failure signal will be subtle enough that users notice before dashboards do.

Context window management and instrumentation for production agentic systems are part of the consulting work I do at hannune.ai.

Why merged cells break table extraction from multi-column PDFs

Tae Kim — Wed, 29 Jul 2026 02:41:03 +0000

PDF tables look simple until you actually extract one. The visual structure in your reader is a rendering artifact. The underlying file has no table object, no row, no cell: just positioned text spans and optional line segments. When a table has merged cells, two or more content spans share the same logical cell, but the PDF records each span at an independent coordinate. The parser sees overlapping positions and has to guess where one cell ends and another begins.

Standard extraction libraries handle this badly, and they handle it badly in different ways.

What the standard tools do

pdfplumber uses horizontal and vertical rules to infer table boundaries, then assigns each text span to the nearest box. When a header cell spans two columns, its text lands in whichever column center is closest. The sibling column gets an empty string. The row count comes out right, but the column attribution is wrong for the merged cell and everything below it in that column.

camelot lattice mode traces the actual line segments to find cells, which makes it better on bordered tables. But merged cells produce intersections at unexpected positions, and the line-tracing algorithm misinterprets them as separate narrow cells. You end up with phantom columns. The stream mode, which ignores borders and uses whitespace, fares worse on multi-column layouts where columns sit close together.

tabula-py (the JVM wrapper around Tabula) splits into lattice and stream modes with the same failure patterns.

Where multi-column layouts make it worse

The problem compounds in multi-column page layouts. Regulatory filings from Korean DART, Japanese EDINET, and Hong Kong HKEX often embed tables inside a two-column page layout. Two tables appear side by side as page columns. A column in the left table overlaps the horizontal coordinate range of a cell in the right table. Position-based heuristics cannot tell which table a text span belongs to without first knowing where each table's boundary is, which is exactly what the parser is trying to find.

I ran into this on balance sheet tables from DART filings: Korean text, two columns on the page, the first containing account names and the second containing values for two reporting periods in adjacent sub-columns. A merged header cell labeled "Current Period" and another labeled "Prior Period" each spanned two numeric sub-columns. pdfplumber placed all four value columns correctly but dropped the period labels. camelot lattice created six columns where there were four, adding ghost columns at the merge intersection points.

What actually worked

The approach that reduced the error rate was two-stage.

First, check whether the table has explicit line segments. If it does, trace the lattice manually and mark every merge point explicitly. A cell spanning columns 2 and 3 gets tagged as a merge, not split into phantom cells. If the table is borderless, fall back to whitespace-based column detection, but seed the column boundaries from the page-level column separator, not from the table's own whitespace. That separator appears as a consistent horizontal gap across the full page height, visible even when individual table rows have variable internal spacing.

For merged cell attribution, the heuristic I settled on is propagation: a header cell spanning columns N through M assigns its text to column N and marks columns N+1 through M as inherited from N. Values in those sub-columns are tagged with the inherited header during downstream processing.

This is not a general solution. Hierarchical merges, where a cell spans both rows and columns, still misfire on documents where the rendering is too compact. The challenge is that row-spanning merges appear as text positioned between two horizontal lines with no structural signal that the text covers more than one row. A parser that does not store the merge map as a first-class object loses that information before it can propagate it.

The underlying limit

The PDF format is the root constraint. Tables are rendered, not encoded. The format specification has no native concept of a merged cell. What you see as a single spanning cell is typically one text element positioned in the visual center of the merged area, with line segments drawn around it. A compliant PDF renderer knows to draw that correctly. An extractor working from coordinate data has to reconstruct the intent from geometry, and geometry is ambiguous at merge points.

The only reliable path I found is to not treat extraction as a pure parsing problem. The geometry pass identifies candidates. A small verification step checks whether the extracted numeric columns sum to stated totals in the document. Mismatches flag cells that were likely misfiled during extraction. That check is separate from the extraction itself, but it is the part that surfaces the errors the geometry heuristics miss.

The full merge detection approach, including how the verification pass catches misfiled cells, is at https://hannune.ai/blog/merged-cells-pdf-table-extraction

I built a PDF table checker that verifies arithmetic in any PDF without a template

Tae Kim — Mon, 27 Jul 2026 02:39:11 +0000

The problem I kept running into: a financial report where a subtotal on page 8 is wrong, but nobody spots it until the numbers get aggregated into a spreadsheet and something doesn't reconcile.

I wanted a tool that could ingest any PDF with tables and tell me which totals don't add up. The constraint: no template, no column mapping UI, no server. Every byte in the browser.

What it does

Upload a PDF. The tool:

Extracts all text items with their x/y coordinates from each page via pdfjs-dist
Clusters items by y-position into rows (within a 4px snap tolerance)
Detects which rows are "total" or "subtotal" rows by keyword matching
For each total row, identifies numeric columns and sums the data rows above it (back to the previous header or total)
Compares computed sum to stated total — flags differences above a 0.1% tolerance

The hard part: coordinate-based table detection

PDF text items don't know they're in a table. The structure is: a stream of (string, x, y, width) tuples. I have to infer table structure purely from spatial relationships.

The approach:

Snap y-coordinates to a 4px grid to handle sub-pixel jitter between lines
Cluster x-positions of numeric cells to find "column anchors" — positions that consistently have numbers
A column anchor that appears in 2+ data rows becomes a verifiable column
For each total row, look up the value at each column anchor and compare to the sum above

This breaks on rotated text, multi-column layouts, and cells that span multiple columns. But it catches the common case — section subtotals in financial reports, grand totals in expense sheets.

Stack

Next.js 14, static export on Cloudflare Workers
pdfjs-dist for text extraction (CDN worker, no bundling)
All detection and arithmetic runs in the browser

Freemium

3 PDFs/month tracked in localStorage. Paid plan unlocks unlimited and CSV export ($9/month).

Live at: https://compare-lab.xyz/pdf-table-checker/

If you work with tabular PDFs regularly and there's a document structure this approach would miss, I'd like to know.

I built a client-side trade document validator — no upload, no server, just the browser

Tae Kim — Mon, 27 Jul 2026 02:38:37 +0000

The problem I kept hitting: an international invoice with a line total that's off by a cent, or an HS code that appears in the commercial invoice but is absent from the packing list. Customs holds are rarely dramatic. They're usually paperwork.

I wanted a tool that would catch these before the documents left the company. The constraint I set: nothing uploaded to a server. Every byte stays in the browser.

What it does

Upload an invoice and a packing list (Excel or PDF). The tool flags:

Sum mismatches — line where qty × unit_price ≠ line total; grand total ≠ sum of lines
HS code conflicts — codes present in one document but not the other
Quantity discrepancies — total quantities that don't match across documents

The parsing challenge

PDF trade documents are the worst case. They're not structured data. The text extraction gives you a stream of strings with no inherent table structure.

The approach I landed on: extract all text, pattern-match for trailing numeric triples (qty, unit price, line total as the last three numbers on a line), and separately extract HS codes using the \d{4}[\.\-]?\d{2}[\.\-]?\d{0,4} pattern. It's imperfect — complex PDFs with merged cells or rotated text break it — but it catches the common case.

Excel is cleaner. SheetJS gives you a proper row/column grid. The trickier part is header detection: the column labeled "HS" in one document might be "Tariff Code" in another. I match a set of synonyms against lowercased, whitespace-stripped header strings.

Stack

Next.js 14, static export, deployed on Cloudflare Workers
xlsx for Excel/CSV
pdfjs-dist with CDN worker (no bundling headache)
All validation logic runs in the browser — parseDocument(), validateSums(), crossValidate()

Free tier

Three document sets per month tracked in localStorage (monthly reset). Paid unlocks unlimited checks and the CSV report download.

Live at: https://compare-lab.xyz/trade-doc-validator/

The source for the validation logic is what I'd most want feedback on. If you deal with trade documents regularly and there's a class of errors this misses, I want to know about it.

Query-Time Entity Disambiguation in Graph RAG: When One Name Means Seventeen Nodes

Tae Kim — Sun, 26 Jul 2026 02:41:58 +0000

The hardest retrieval problem in Graph RAG is not missing data. It is the query that arrives with an ambiguous entity name.

In 2asy.ai, a knowledge graph built on Korean corporate data, "Hyundai" matches seventeen separate nodes. A vector search returns all of them ranked by embedding similarity. A graph traversal needs exactly one starting node. The gap between these two realities is where Graph RAG quietly breaks.

The Setup: Where Disambiguation Fits in the Pipeline

In a standard Graph RAG pipeline, the flow is:

Parse the natural language query
Extract entity mentions
Disambiguate each mention to a specific graph node ← the problem
Traverse the graph from that node
Feed retrieved subgraph into the generation step

Step 3 is where "Hyundai" needs to resolve to either Hyundai Motor Company, Hyundai Engineering & Construction, Hyundai Steel, Hyundai Merchant Marine, or one of the other thirteen. The wrong choice produces a coherent-sounding answer about the wrong company.

from dataclasses import dataclass
from datetime import datetime

@dataclass
class DisambiguationResult:
    node_id: str
    display_name: str
    confidence: float
    signal_breakdown: dict[str, float]

def disambiguate(
    mention: str,
    candidates: list[dict],
    context_terms: list[str],
    query_time: datetime,
) -> DisambiguationResult:
    scores = {}
    for candidate in candidates:
        freq = frequency_prior_score(mention, candidate)
        ctx = context_coherence_score(candidate, context_terms)
        temporal = temporal_activity_score(candidate, query_time)
        scores[candidate["node_id"]] = {
            "frequency": freq,
            "context": ctx,
            "temporal": temporal,
            "combined": freq * 0.35 + ctx * 0.45 + temporal * 0.20,
        }

    best = max(scores, key=lambda nid: scores[nid]["combined"])
    return DisambiguationResult(
        node_id=best,
        display_name=next(c["name"] for c in candidates if c["node_id"] == best),
        confidence=scores[best]["combined"],
        signal_breakdown=scores[best],
    )

Signal 1: Corpus Frequency Prior

Without any query context, the entity that appears most often in the source corpus where the mention is unqualified is the right default.

In the Korean financial news corpus underlying 2asy.ai, "Hyundai" without a qualifier co-occurs with Hyundai Motor documents about two-thirds of the time. That prior is learnable from the corpus and beats a random guess at the disambiguation step even before any query-specific signals are considered.

def frequency_prior_score(mention: str, candidate: dict) -> float:
    """
    Fraction of unqualified mentions of `mention` that refer to this candidate
    in the training corpus. Stored as a per-mention-string dict on the node.
    """
    priors = candidate.get("mention_priors", {})
    return priors.get(mention.lower(), candidate.get("default_frequency_prior", 0.0))

The prior is stored on each node at graph-build time. It is not recomputed at query time — that would be slow. What gets stored is the fraction of unambiguous co-occurrences from the corpus. A candidate with a prior of 0.65 means: historically, this name referred to this entity 65% of the time it appeared without additional qualifiers.

Signal 2: Query Context Coherence

The mention rarely arrives in isolation. If the same query mentions "battery technology" or "manufacturing capacity," that co-occurrence signal shifts probability mass toward Hyundai Motor. If the query mentions "construction permits" or "project bids," Hyundai Engineering moves up.

This is a co-occurrence statistic between the query terms and each candidate's entity description:

import math
from collections import Counter

def context_coherence_score(
    candidate: dict,
    context_terms: list[str],
    smoothing: float = 1.0,
) -> float:
    """
    Pointwise mutual information between context_terms and candidate description tokens.
    Higher score = more context terms co-occur with this entity in training data.
    """
    if not context_terms:
        return 0.5  # neutral when no context available

    description_tokens = set(candidate.get("description_tokens", []))
    cooccurrence_counts = candidate.get("cooccurrence_counts", {})
    total_docs = candidate.get("total_docs_in_corpus", 1)

    pmi_sum = 0.0
    for term in context_terms:
        p_term = cooccurrence_counts.get(term, 0) / total_docs
        p_entity = candidate.get("doc_count", 0) / total_docs
        p_joint = cooccurrence_counts.get(f"joint:{term}", 0) / total_docs

        if p_joint > 0 and p_term > 0 and p_entity > 0:
            pmi = math.log2(p_joint / (p_term * p_entity) + smoothing)
            pmi_sum += max(0.0, pmi)

    return min(1.0, pmi_sum / (len(context_terms) + smoothing))

In practice, storing full co-occurrence counts per candidate is expensive. We store the top-500 co-occurring terms per entity node, which covers the vast majority of real query patterns.

Signal 3: Temporal Suppression

A historical subsidiary with a valid_to date should rank below currently-active entities for queries that do not specify a historical time window. The graph already has the structure to express this — active vs. inactive entity status — but the query layer has to use it.

def temporal_activity_score(candidate: dict, query_time: datetime) -> float:
    """
    Returns 1.0 if the entity is active at query_time, 0.1 if inactive.
    Entities without a valid_to are assumed ongoing.
    """
    valid_from = candidate.get("valid_from")
    valid_to = candidate.get("valid_to")

    if valid_to is not None:
        active = valid_from <= query_time <= valid_to
    else:
        active = valid_from <= query_time if valid_from else True

    return 1.0 if active else 0.1

The weight of 0.1 (not zero) lets historical entities still be retrieved if the other signals are strong enough — useful when a query explicitly references a past entity. But for the common case of a present-tense query, a deactivated entity is strongly suppressed.

The Failure Mode: Silent Wrong Answers

When disambiguation selects the wrong candidate, the failure is invisible. The graph traversal returns a coherent subgraph, the generation step produces a confident-sounding answer, and nothing in the output signals that retrieval started from the wrong node.

The user who asked about "Hyundai's battery supply chain" gets a detailed answer about Hyundai Engineering's infrastructure projects instead. The answer is internally consistent. It is just about the wrong company.

This is worse than a retrieval miss. A retrieval miss produces an "I don't have information about that" response that the user recognizes as incomplete. A disambiguation error produces a fluent, confident, wrong answer.

The Fix: Make the Disambiguation Decision Visible

The disambiguation machinery is already doing the work of picking a candidate. The only change is surfacing that decision before the answer is rendered.

def rag_query(user_query: str, graph, llm) -> str:
    mentions = extract_entity_mentions(user_query)
    resolved = []

    for mention in mentions:
        candidates = graph.get_candidates(mention)
        if len(candidates) == 1:
            resolved.append(candidates[0])
            continue

        result = disambiguate(
            mention=mention,
            candidates=candidates,
            context_terms=extract_context_terms(user_query, exclude=mentions),
            query_time=datetime.utcnow(),
        )
        resolved.append(result)

        # Surface the decision before answering
        if result.confidence < 0.85:
            print(f"[disambiguation] '{mention}' → {result.display_name} "
                  f"(confidence: {result.confidence:.0%}, "
                  f"top alternative: {get_second_best(candidates, result.node_id)})")

    subgraph = graph.traverse(start_nodes=[r.node_id for r in resolved])
    return llm.generate(context=subgraph, query=user_query)

Showing [disambiguation] 'Hyundai' → Hyundai Motor Company (confidence: 71%, top alternative: Hyundai E&C) before the answer converts a silent failure into something the user can catch and correct.

The threshold of 0.85 matters. Above that, the disambiguation is confident enough that surfacing it would be noise. Below it, the user should know the system made a choice.

Why This Matters More Than Retrieval Precision

Most Graph RAG evaluation focuses on retrieval recall — whether the right documents or subgraph edges are found. Disambiguation happens before retrieval. An error there means retrieval runs correctly on the wrong entity's neighborhood.

The node count in a well-maintained corporate knowledge graph is in the hundreds of thousands. Unqualified company name mentions resolve to the right entity most of the time — until they do not, and then the failure is invisible. Treating disambiguation as a first-class step, with explicit confidence scoring and user-visible decisions, is what keeps it from becoming a silent quality drain.

Building 2asy.ai, a knowledge graph for Korean corporate intelligence. More at hannune.ai.

Cover: Y (@yi2026) on Unsplash

Tool Schema Drift: The Silent Failure Mode in Production Agentic Systems

Tae Kim — Wed, 22 Jul 2026 02:36:39 +0000

The most common agentic system failure I encounter in production is not a bad prompt. It is not a context overflow. It is a tool that changed without its registration changing.

I have seen this cause weeks of debugging in systems that were working fine until they weren't — and because the failure is silent, teams often spend time looking at the model first.

What Tool Schema Drift Looks Like

Most agentic frameworks let you register tools with a name, a description, and a JSON schema for parameters. The model reads the description to decide when to call the tool. It reads the schema to know what parameters to send.

# Initial registration — looks fine
tools = [
    {
        "name": "search_entities",
        "description": "Search the entity registry by name. Returns a list of matching entities.",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search term"}
            },
            "required": ["query"]
        }
    }
]

Six months later, the underlying function has changed. The team added a required entity_type filter after learning that unfiltered searches returned too many results. They updated the implementation. They forgot to update the registration.

# Current implementation — mismatched with registration
def search_entities(query: str, entity_type: str) -> list[dict]:
    # entity_type is now required — "company", "person", "location"
    return registry.search(query=query, type=entity_type)

Now the model calls search_entities with {"query": "Samsung"}. The function raises a TypeError because entity_type is missing. Or worse — if you have a default fallback somewhere upstream — it silently runs with the wrong parameters and returns results the model was not expecting.

The model sees a response. It generates text from whatever came back. No exception fires. The output just drifts from what it should have been.

Why This Is Hard to Catch

The failure mode is subtle for two reasons.

First, evals usually test output quality, not the call-response cycle. If your eval checks whether the final answer is correct, a tool that silently misbehaves can still produce plausible-looking output — especially if the task has any ambiguity in what "correct" means.

Second, the description is what the model acts on, not the schema. A schema mismatch causes a runtime error. A description mismatch causes behavioral drift — the model calls the tool in situations where it should not, or does not call it when it should, and there is no error signal.

Both failure modes happen in production. Neither shows up in type checking or unit tests.

Three Checks That Catch Drift Before Production

1. Validate the response shape, not just the call shape

Most frameworks validate that the model produced a well-formed tool call (correct JSON, required fields present). Fewer validate that the tool's response matched what the model was told to expect.

from pydantic import BaseModel, ValidationError
from typing import Any

class EntitySearchResult(BaseModel):
    entity_id: str
    name: str
    entity_type: str
    confidence: float

def call_tool_with_validation(tool_name: str, args: dict) -> Any:
    raw_response = dispatch_tool(tool_name, args)

    # Validate response matches what the model expects
    if tool_name == "search_entities":
        try:
            validated = [EntitySearchResult(**item) for item in raw_response]
            return validated
        except ValidationError as e:
            # Response shape changed — surface this immediately
            raise ToolResponseSchemaError(
                f"Tool '{tool_name}' returned unexpected shape. "
                f"Registration is out of sync with implementation.\n{e}"
            )

    return raw_response

This makes the mismatch loud. A ToolResponseSchemaError is unmistakable. A silently wrong answer is not.

2. Version the tool description alongside the implementation

Treat the description as part of the implementation contract, not as documentation.

The practical pattern: keep tool schemas in a registry file under version control, and make updating the registry a required step when any tool function signature or return shape changes. Code review that accepts a function change without a registry update is accepting a potential drift.

# tools/registry/v2.py — versioned alongside implementation
TOOL_REGISTRY = {
    "search_entities": {
        "version": "2.0.0",
        "description": (
            "Search the entity registry by name and type. "
            "Returns matching entities with confidence scores. "
            "entity_type must be one of: company, person, location."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "entity_type": {
                    "type": "string",
                    "enum": ["company", "person", "location"]
                }
            },
            "required": ["query", "entity_type"]
        },
        "response_schema": {
            "type": "array",
            "items": {
                "type": "object",
                "required": ["entity_id", "name", "entity_type", "confidence"]
            }
        }
    }
}

If the interface changes in a backward-incompatible way — new required parameter, different return shape — give it a new name (search_entities_v2) rather than silently updating the existing entry. Agents that depended on the old interface continue to work until they are explicitly migrated.

3. Add canary evals that cover the full call-response cycle

A single eval prompt per tool is enough to catch schema drift in CI.

# tests/evals/test_tool_contracts.py
import pytest

def test_search_entities_contract():
    """
    Canary eval: triggers the tool and validates the full cycle.
    If search_entities changes its interface, this breaks before production does.
    """
    agent_response = run_agent(
        prompt="Find all company entities named Samsung in the registry.",
        tools=TOOL_REGISTRY,
    )

    # Check the tool was called
    tool_calls = extract_tool_calls(agent_response)
    search_calls = [c for c in tool_calls if c["name"] == "search_entities"]
    assert len(search_calls) > 0, "Agent did not call search_entities"

    # Check the call used the current schema
    for call in search_calls:
        assert "entity_type" in call["args"], (
            "Agent called search_entities without entity_type — "
            "tool description may be out of sync with schema"
        )

    # Check the response matched expected shape
    assert agent_response.tool_errors == [], (
        f"Tool returned unexpected shape: {agent_response.tool_errors}"
    )

This eval does not test whether the final answer is good. It tests whether the tool call cycle completed without a schema mismatch. That is exactly what breaks first when a tool drifts.

The Root Cause

The reason this happens repeatedly is that tool descriptions live outside the usual code review discipline. They are strings in a config or a dict literal. Nobody has a linter that flags "the function signature changed but the description did not."

The fix is not complicated. It is a process discipline problem more than a technical one. The registration is the contract between the model and the implementation. Versioning it like a contract — with change control, backward-compatibility rules, and automated validation — closes the gap that produces silent drift.

The agent is not wrong when it mishandles an unexpected tool response. It was not told the tool changed. If you treat the registration as the contract, you start maintaining it like one.

I work on entity resolution and agentic infrastructure at er-api.hannune.ai. If you have seen this failure mode in your own systems, curious what the trigger was.

When the same incident becomes five separate events in your graph

Tae Kim — Tue, 21 Jul 2026 02:37:53 +0000

A single corporate event in East Asia rarely shows up once. A merger gets filed at OpenDART in Korean, reported by EDINET in Japanese a day later, picked up by a Hong Kong exchange notice in traditional Chinese, and summarized in an English wire story. Four documents, one real-world incident. If your pipeline extracts events document by document, you end up with four Event nodes in the graph that are really the same thing.

That is the situation I hit in the knowledge graph behind 2asy.ai, and the fix turned out to be a different problem than I first assumed.

Event coreference is not entity resolution

It is easy to conflate the two. Entity resolution answers "is this Samsung the same as that Samsung," matching company or person names across languages and spellings. I already solve that upstream with a cross-lingual entity resolution pass, so by the time events land in the graph, "삼성전자" and "Samsung Electronics" point to one canonical entity ID.

Event coreference is the next layer. Even with entities unified, the events themselves stay split. Two documents can both reference the same canonical Samsung entity and still describe what looks like two separate filings, because each was extracted in isolation. Coreference asks a harder question: are these two event records describing the same incident, or two genuinely different things that happen to share an actor?

Why the graph splits events in the first place

The graph builder follows a one-document, one-LLM-call rule. Each article or disclosure is read once, in its own language, and the model emits the events it finds in that document alone. That is deliberate. Extraction stays auditable, cost per document stays bounded, and I never hand one giant prompt a whole day of news and watch it hallucinate links between stories that were never connected.

The cost is duplication. The builder has no memory of what another document said, so the same merger becomes a Korean Event node, a Japanese one, and an English one. To a reader traversing the graph that looks like three mergers. There was one.

The naive fix breaks on East Asian data

The obvious shortcut is to dedup events by string similarity on their summaries. It fails here for two reasons. First, the summaries are in different languages, so token overlap between a Korean summary and an English one is close to zero even when they describe the identical event. Second, even within one language, two genuinely distinct events involving the same companies, say a supply contract and a separate lawsuit filed the same week, can share most of their vocabulary. String matching cannot tell coincidental overlap from real coreference.

The two-stage approach I settled on

I built a small standalone resolver that runs read-only against the graph and emits clusters as JSON. It never writes back to Neo4j. The design has two stages on purpose: a cheap rule-based filter, then a targeted LLM judgment.

Stage one pulls recent Event nodes inside a time window and forms candidate pairs without calling any model. A pair survives only if it clears three gates: the events share at least one canonical entity, their date buckets fall within seventy-two hours, and they either carry the same event type or their summaries clear a Jaccard token-overlap threshold. Each surviving pair gets a similarity score and the list is sorted so the strongest candidates come first. This stage exists to keep the LLM bill bounded. With a per-run cap on calls, you judge the top candidates by score and skip the long tail of weak pairs.

Stage two hands each candidate pair to a small model with both summaries, both entity lists, and both date buckets, and asks for one of three labels: same, related, or distinct. I keep the three-way distinction on purpose. Collapsing related into same would merge a contract and a lawsuit between the same two firms; collapsing it into distinct would scatter genuine follow-on coverage. Only a "same" verdict counts as a merge edge.

Stage three is plain union-find. Every "same" verdict unions the two event IDs, and the connected components become clusters. Any component with a single member gets dropped, since a cluster of one says nothing about cross-document coverage. Each surviving cluster record carries its member event IDs, the union of their entities, the source documents, the time range, and the model's one-line reason for every merge it made. I kept that reason log so I can read back why two events got joined when a cluster looks wrong.

Design decisions that mattered

The one-LLM-call-per-pair rule mirrors the one-document rule in the builder. It is slower than batching a cluster into a single prompt, but it keeps each judgment independent and easy to inspect, and it stops the model from inventing transitive links it was never asked about. Transitivity comes from union-find on discrete yes-or-no verdicts, not from the model reasoning over a whole group at once.

The seventy-two-hour window is a blunt instrument, and it is the part I trust least. Cross-border coverage of the same incident usually lands inside three days, but slow regulatory follow-ups can arrive a week later and get missed. Widening the window quadratically inflates the candidate-pair count, which is exactly what the rule filter is there to contain. That tension between recall and cost is unresolved, and for now I chose the cheaper side.

The whole thing runs against a read-only Neo4j connection and outputs a file. Nothing touches the operational graph. Promoting a cluster ID back onto the Event nodes is a separate, deliberate step, not something a nightly job does on its own.

Where this sits

Right now the resolver is an offline pass that produces clusters for review. It is not yet a wired-in stage of the daily pipeline. The next move is letting the content layer read cluster IDs, so a brief can say "covered across four filings in three jurisdictions" instead of treating each filing as its own event. The point was never a tidier graph. It was a reader who sees that one incident got reported four times, in four languages, and can trust that the graph already worked that out.

Cross-Lingual Entity Resolution: Why Your Knowledge Graph Has Four Samsungs

Tae Kim — Sun, 19 Jul 2026 02:38:38 +0000

Cross-lingual entity resolution is where knowledge graphs quietly break down.

I ran into this building er-api, a multilingual entity resolution service for Korean, Japanese, Chinese, and English corporate data.

The problem is deceptively simple: Samsung Electronics appears as four different strings across source documents.

Korean: 삼성전자
English: Samsung Electronics
Japanese: サムスン電子
Chinese: 三星电子

Each string hashes to a different node in the graph. A query for Samsung Electronics misses three-quarters of the data unless you happen to use the exact string form the graph was built with. One real company becomes four phantom entities.

Why String Matching Fails

The instinct is to use fuzzy string matching — Levenshtein distance, cosine similarity on character n-grams. This works for typos and abbreviations within the same script. It fails completely across scripts.

"Samsung Electronics" and "サムスン電子" share no character sequences. No string distance metric will recognize them as the same entity without preprocessing.

Step 1: Transliterate to a Canonical Form

The first step is converting all strings to a common script before comparison.

import jaconv  # Japanese: katakana/hiragana to romaji
from hanja import translate as hanja_to_hangul
from hangul_romanize import Transliterate
from hangul_romanize.rule import RevisedRomanization

transliterator = Transliterate(RevisedRomanization())

def canonical_form(text: str, source_lang: str) -> str:
    """Convert a name to a canonical Latin-ish form for comparison."""
    if source_lang == "ja":
        text = jaconv.kata2hepburn(jaconv.hira2kata(text))
    elif source_lang == "zh":
        text = hanja_to_hangul(text)
        source_lang = "ko"

    if source_lang == "ko":
        text = transliterator.translit(text)

    return text.lower().strip()

After transliteration:

삼성전자 → "samsung jeonja"
Samsung Electronics → "samsung electronics"
サムスン電子 → "samsung denshi"
三星电子 → "삼성전자" → "samsung jeonja"

Not identical, but close enough that a similarity metric can catch them. Transliteration alone collapses a large fraction of duplicates before any similarity computation runs.

Step 2: Blocking Keys

Comparing all n-squared candidate pairs across four languages at scale is not viable. For a corpus of 100k entities across four languages, that is 10 billion pairs.

Blocking reduces candidates to a manageable set by only comparing pairs that share a blocking key — a coarse-grained signature that likely-duplicate entities share.

import re

def blocking_key(name: str, lang: str) -> str:
    """
    Generate a blocking key for grouping candidate pairs.
    Strips legal suffixes, normalizes, takes first 4 chars.
    """
    canonical = canonical_form(name, lang)

    legal_suffixes = [
        r"\b(co|corp|inc|ltd|llc|plc|gmbh|ag|sa|bv)\b\.?",
        r"\b(주식회사|kabushiki kaisha|kk|株式会社|有限公司)\b",
    ]
    for pattern in legal_suffixes:
        canonical = re.sub(pattern, "", canonical, flags=re.IGNORECASE)

    canonical = re.sub(r"[^a-z0-9]", "", canonical)
    return canonical[:4]

All entities that share the same blocking key enter the same candidate pool for pairwise comparison. Entities in different blocks never get compared.

Step 3: Per-Language-Pair Similarity Thresholds

Once you have candidate pairs within a block, you need a similarity score. Splink's EM estimator works well here — it learns match probabilities from the data without requiring fully labeled examples.

But the threshold for what counts as a match is not the same across language pairs.

Korean-English pairs tolerate more abbreviation variance. "Samsung C&T" vs. "Samsung C&T Corporation" — same entity, different abbreviation patterns. A Korean-English threshold of 0.75 captures this without over-merging.

Japanese-Chinese pairs are tighter. The transliteration paths for Japanese and Chinese overlap less cleanly than Korean-Latin paths. A threshold of 0.85+ is safer here.

LANG_PAIR_THRESHOLDS = {
    ("ko", "en"): 0.75,
    ("ko", "ja"): 0.80,
    ("ko", "zh"): 0.82,
    ("en", "ja"): 0.78,
    ("en", "zh"): 0.80,
    ("ja", "zh"): 0.85,
}

def threshold_for_pair(lang_a: str, lang_b: str) -> float:
    key = tuple(sorted([lang_a, lang_b]))
    return LANG_PAIR_THRESHOLDS.get(key, 0.80)

The Hardest Part: Labeled Training Data

Splink's EM estimator can run unsupervised, but performance is better with labeled examples.

For Korean-English, there is labeled data available from Korean financial filings — companies are required to report both the Korean and English names in disclosure documents. A financial data scraper gives you positive pairs cheaply.

For Korean-Chinese and Japanese-Chinese, there is almost no open labeled data. What actually worked:

Generating negative pairs by sampling clearly distinct companies from different industry sectors — any two companies with no overlapping industry codes and no name similarity are almost certainly different entities.
Bootstrapping positive pairs from a subset of companies with known cross-lingual identifiers (Dun & Bradstreet DUNS numbers, Bloomberg tickers) where the same company appears in all four language corpora.
Manual annotation for the ambiguous middle tier — about 2,000 pairs reviewed by a bilingual Korean-Chinese speaker.

It is not elegant. But the EM estimator is sensitive enough to training data quality that this effort pays back in precision.

The Result: The Graph Shrinks

After running cross-lingual ER on the initial corpus:

Node count decreased by approximately 18%. Four phantom nodes collapsing to one real node, repeated across the corpus.
Graph RAG retrieval precision for cross-lingual queries improved significantly. Queries for a company in English now correctly retrieve documents from Korean, Japanese, and Chinese sources.
False positive rate stayed low because blocking plus per-language-pair thresholds prevented over-merging.

The graph shrinks after you apply this. Retrieval quality improves because queries no longer split across phantom duplicates.

Building er-api, a multilingual entity resolution service for Korean, Japanese, Chinese, and English corporate data. More at hannune.ai.

Cover: Alina Grubnyak on Unsplash