How we did entity resolution with a vector DB and Argon

This article was originally published on the Actian blog.

In this tutorial, you'll build an entity resolution pipeline in Python that finds the same product listed twice across two catalogs. Actian VectorAI DB blocks each record to its 10 nearest neighbors, a similarity threshold settles the clear-cut pairs, and Gemini 4 Argon decides only the uncertain band, returning a match decision with a one-sentence reason. Everything except the Argon calls runs on your own machine.

On the Abt-Buy benchmark, the threshold alone reached an F1 of 0.813. With Argon on the uncertain pairs, F1 rose to 0.915 and precision to 0.998, for $0.76 per 1,000 records. All code is in the companion repo. If you're in a hurry, follow steps 1 to 3 and skip to step 8.

Results at a glance

Stage	Abt-Buy result
Blocking	Kept 10,810 of 1,180,452 possible pairs (99.08% removed) and 99.0% of true matches
Threshold only	F1 0.813
With Argon on the uncertain band	F1 0.915, precision 0.998
Sent to Argon	13.2% of candidate pairs, 1,001 of 7,570 on the test split
Cost	$1.41 at OpenRouter's charges, or $0.76 per 1,000 records in the test split

Prerequisites

• You need Docker with the Compose plugin.
• You need uv, the Python package manager, which installs a compatible Python for you.
• You need Ollama to run the embedding model locally.
• You need a Gemini 4 Argon key, either an OpenRouter key with a few dollars of credit or a Google AI Studio or Google Cloud key with billing turned on. Argon has no free tier, and the Abt-Buy run cost $1.41.
• You need about 3 GB of free disk for the database image and the embedding model. Everything runs on a CPU.

VectorAI DB needs no separate install, because step 1 runs Community Edition in Docker. Each benchmark fits under its 5,000-vector cap, one at a time. Install uv if you don't have it.

# macOS or Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
irm https://astral.sh/uv/install.ps1 | iex

How the pipeline works

Entity resolution, also called record linkage or entity matching, runs in three steps. Blocking cuts the pairs down to candidates, comparison scores each candidate, and matching makes the final call. This build gives each step its own tool.

• VectorAI DB does the blocking, returning each record's nearest neighbors from the other table so the pipeline scores 10 candidates per record.
• EmbeddingGemma, run locally through Ollama, turns each record into a 768-dimensional vector.
• A similarity threshold auto-matches confident pairs and rejects clear non-matches.
• Gemini 4 Argon decides the uncertain band in between and gives a one-sentence reason.
• resolve.py connects them and scores the results against the benchmark's known matches.

Records are embedded and blocked in VectorAI DB, and the similarity score routes each candidate pair to an auto-match, a rejection, or Argon

The examples use Abt-Buy from the Leipzig benchmark collection, with 1,081 Abt products, 1,092 Buy products, and a gold file of 1,097 true matches. Comparing every Abt record with every Buy record would mean 1,180,452 pairs.

1. Start VectorAI DB

Clone the companion repo and move into the project folder. Every command from here on runs in this folder.

git clone https://github.com/actian-devs/developer-content.git
cd developer-content/blog/entity-resolution-vectorai

The repo's docker-compose.yml starts VectorAI DB. The ACTIAN_VECTORAI_ACCEPT_EULA=YES line accepts Actian's license for you, so read it before you start the container.

services:
  vectorai:
    image: actian/vectorai:1.0.2
    container_name: vectorai
    ports:
      - "6573:6573" # REST
      - "6574:6574" # gRPC (Python SDK)
      - "6575:6575" # Local UI
    volumes:
      - ./vectorai_data:/var/lib/actian-vectorai
    environment:
      - ACTIAN_VECTORAI_ACCEPT_EULA=YES
    restart: unless-stopped

docker compose up -d
docker ps

The container should show Up. The Python SDK talks to it over gRPC on port 6574.

The Community Edition's 5,000-vector cap covers the whole server, and deleted vectors kept counting against it until the container restarted. The harness drops its own old collections. If it still reports the cap, run docker compose restart vectorai.

2. Pull the embedding model

EmbeddingGemma is about 620 MB.

ollama pull embeddinggemma

Ollama serves it on localhost:11434, and the harness calls it there. Nothing leaves your machine at this stage.

3. Install Python dependencies

The repo already includes pyproject.toml, and its uv.lock pins the exact versions used, such as actian-vectorai-client 1.0.3 and google-genai 2.31.0.

[project]
name = "entity-resolution-vectorai"
version = "0.1.0"
description = "Entity resolution with VectorAI DB blocking and Gemini 4 Argon on the ambiguous band"
requires-python = ">=3.11,<3.14"
# `uv run resolve.py` reads this, creates .venv and installs these on first run
dependencies = [
    "actian-vectorai-client",  # VectorAI DB client (gRPC)
    "google-genai",            # Gemini API, for the Argon pass
    "pandas",
    "numpy",
    "pydantic",                # Argon's structured-output schema
    "requests",                # benchmark download
    "matplotlib",              # score-distribution chart
]

[project.optional-dependencies]
st = ["sentence-transformers"]   # in-process embedding models: uv sync --extra st, then --embed-backend sentence-transformers

uv sync

From here on, start each command with uv run. You don't need to activate an environment.

4. Load and embed the records

The source table goes into each point's payload, because step 5 filters on it, so an Abt record is only ever compared with Buy records. Embedding is the slowest step on a CPU, so the harness caches the vectors, and reruns skip it. The code below is the core of load() in er/data.py, embed()in er/embed.py, and load_records() in er/block.py.

import numpy as np
import requests
from actian_vectorai import Distance, PointStruct, VectorAIClient, VectorParams

def to_text(row, fields):
    # "name: Sony Bravia 40in LCD TV | price: $899.99" (empty fields are skipped)
    return " | ".join(f"{f}: {row[f].strip()}" for f in fields if row[f].strip())

def embed(texts, model="embeddinggemma", url="http://localhost:11434", batch=64):
    out = []
    for i in range(0, len(texts), batch):
        r = requests.post(f"{url}/api/embed", json={"model": model, "input": texts[i:i + batch]}, timeout=600)
        r.raise_for_status()
        out.extend(r.json()["embeddings"])
    v = np.asarray(out, dtype=np.float32)
    return v / np.linalg.norm(v, axis=1, keepdims=True)   # unit length, so cosine score = dot product

vectors = embed([text for _, _, text in records])          # records: (source, record_id, text)

with VectorAIClient("localhost:6574") as db:
    if db.collections.exists("er_abt_buy"):   # start clean, so the step can be rerun
        db.collections.delete("er_abt_buy")
    db.collections.create("er_abt_buy", vectors_config=VectorParams(
        size=vectors.shape[1], distance=Distance.Cosine))
    points = [PointStruct(id=i, vector=v.tolist(), payload={"source": src, "record_id": rid, "text": t})
              for i, ((src, rid, t), v) in enumerate(zip(records, vectors))]
    for start in range(0, len(points), 500):   # upsert in batches
        db.points.upsert("er_abt_buy", points[start:start + 500])

5. Block with VectorAI DB

The blocking query runs once per Abt record and asks for its K nearest Buy records, with a must filter on the source field so the search only returns records from the other table. Zeakis et al. evaluated 12 language models on 17 entity resolution benchmarks, including how well their embeddings block. This is the core of neighbors () in er/block.py.

from actian_vectorai import Field, FilterBuilder, VectorAIClient

K = 10
TOTAL_RECORDS = len(records)   # Abt + Buy: a limit this high returns every Buy record
abt_ids = [rid for src, rid, _ in records if src == "abt"]
abt_vectors = vectors[:len(abt_ids)]   # records lists every Abt record first
other_source = FilterBuilder().must(Field("source").eq("buy")).build()

def search(db, vectors, limit):
    out = []
    for i in range(0, len(vectors), 100):   # search_batch takes up to 100 queries per call
        out += db.points.search_batch("er_abt_buy", [
            {"vector": v.tolist(), "limit": limit, "filter": other_source, "with_payload": True}
            for v in vectors[i:i + 100]])
    return out

ranked, todo, limit = {}, list(range(len(abt_ids))), K * 10   # over-fetch, then keep the top K
with VectorAIClient("localhost:6574") as db:
    while todo:
        short = []
        for i, hits in zip(todo, search(db, abt_vectors[todo], limit)):
            ranked[abt_ids[i]] = sorted(((h.payload["record_id"], h.score) for h in hits),
                                        key=lambda x: -x[1])[:K]
            if len(hits) < K and limit < TOTAL_RECORDS:   # stop once the limit covers the table
                short.append(i)
        todo, limit = short, min(limit * 4, TOTAL_RECORDS)   # retry any record still short

Over-fetch when you filter

On VectorAI DB 1.0.2 with the 1.0.3 Python client, a search_batch call with limit=10 and the source = buy filter came back short for 1,075 of the 1,081 Abt records, with 3.9 results on average. The filtered results looked like the Buy records among roughly the limit nearest records overall. The predicate filters guide says filters apply during the search, so treat this as what 1.0.2 did on this collection.

Raising hnsw_ef to 256 changed nothing, and 1.0.2 rejects adding a payload index to an existing collection. Asking for 10 × K and keeping the top K fixed it. Every record got its 10 candidates, and 99.95% of them matched an exact brute-force search. On customer or supplier data, the samemust clause can rule out a pair completely, such as a different country. Raise the limit until every record gets its K, because the more records a filter excludes, the fewer of the nearest records pass it.

Read the reduction ratio and pair completeness together

The reduction ratio is the share of all pairs that blocking removed. Pair completeness is the share of true matches that survived blocking, and it caps recall for the whole pipeline.

import pandas as pd

candidates = {(abt_id, buy_id): score for abt_id, hits in ranked.items() for buy_id, score in hits}
n_buy = len(records) - len(abt_ids)
gold = set(map(tuple, pd.read_csv("data/abt-buy/abt_buy_perfectMapping.csv").astype(str).values))
rr = 1 - len(candidates) / (len(abt_ids) * n_buy)
pc = len(gold & candidates.keys()) / len(gold)

K	Candidate pairs	Reduction ratio	Pair completeness
1	1,081	99.91%	81.9%
3	3,243	99.73%	94.5%
5	5,405	99.54%	97.0%
10	10,810	99.08%	99.0%
20	21,620	98.17%	99.7%

K = 10 removed 99.08% of the comparison space and kept 99.0% of the true matches. K = 20 adds 0.7 points of pair completeness for twice the candidates. Pick the smallest K whose pair completeness you can live with.

One square per 1,000 pairs on Abt-Buy. VectorAI DB kept 10,810 of the 1,180,452 possible pairs, and Argon read 1,001 of them

6. Score the candidates and set the band

The baseline is a best-match threshold, where each Abt record's nearest candidate is a match if its similarity clears one cutoff. On the test split, the best cutoff was 0.562, with precision 0.820, recall 0.806, and F1 0.813. One global cutoff over all 10 candidates per record scored only 0.522.

All 136 false matches sit inside the band, between 0.56 and 0.84, mixed in with true matches

The harness holds out 30% of the Abt records as a development split to set the cutoff and both band edges, and every result is reported on the remaining 70%.

• The upper edge is the lowest score above which best matches reach 95% precision on the development split. Best matches above it are auto-matched.
• The lower edge is the highest score below which at most 5% of true matches fall, capped at the baseline's cutoff. Pairs below it are rejected.
• Near-ties are second- or third-ranked candidates within 0.03 of their record's best, and they also enter the band. 18% of true matches aren't their record's top candidate.

This code comes from er/bands.py, which also holds the helpers rank_and_gap(), best_match(), and best_threshold().

import numpy as np

def band_edges(cand, gold, target_precision, max_miss):
    pairs = sorted(cand.items(), key=lambda kv: kv[1], reverse=True)
    s = np.array([score for _, score in pairs])
    y = np.array([p in gold for p, _ in pairs], dtype=float)
    precision_above = np.cumsum(y) / np.arange(1, len(y) + 1)
    ok = np.where(precision_above >= target_precision)[0]
    upper = s[ok.max()] if len(ok) else s[0] + 1e-9   # nothing reaches the target: auto-match nothing
    missed_below = (y.sum() - np.cumsum(y)) / y.sum()   # share of matches scoring below each position
    lower = s[np.where(missed_below <= max_miss)[0].min()]
    return float(min(lower, upper)), float(upper)

def near_tie_band(cand, info, lower, upper, max_rank=3, max_gap=0.03):
    band = []
    for pair, score in cand.items():
        rank, gap = info[pair]   # rank among the record's candidates, distance below its best
        if score < lower:
            continue
        if (rank == 1 and score < upper) or (2 <= rank <= max_rank and gap <= max_gap):
            band.append(pair)
    return band

lower, upper = band_edges(dev_best, dev_gold, target_precision=0.95, max_miss=0.05)
lower = min(lower, best_cutoff)   # never reject a pair the baseline accepts
band = near_tie_band(test_candidates, info, lower, upper)

The edges landed at 0.562 and 0.841. The band held 1,001 of the 7,570 test candidates, and 206 of the baseline's 279 errors sat inside it. If Argon decided every band pair correctly, recall could rise from 0.806 to 0.897. Measure that share on your own data first, because errors outside the band can't be fixed by a second pass.

The hard pairs differ by a detail an embedding blurs. The Sharp LC37D44U scored 0.832 against an Abt listing for the LC37D64U, while the correct LC-37D64U scored 0.814 as the second candidate. Here's the first pair as the embedding model and Argon see it.

Record A (Abt): name: Sharp AQUOS 37' Black LCD HDTV - LC37D64U | description: Sharp AQUOS 37' Black LCD HDTV - LC37D64U/ 1920 x 1080p Pixel Resolution/ Up To 60,000 Hours Lamp Life/ Built-In ATSC/QAM/NTSC Tuner/ 10,000:1 Dynamic Contrast Ratio/ 4ms Response Time/ 4-Wavelength Backlight System/ Black Finish

Record B (Buy): name: Sharp AQUOS LC37D44U 37' LCD TV - LC37D44U | description: Sharp AQUOS LC-37D44U - 37' Widescreen LCD HDTV - 7500:1 Dynamic Contrast Ratio - 6ms Response Time | manufacturer: Sharp

7. Send the band to Argon

Each band pair goes to Argon and comes back as structured output, a match or no-match with a one-sentence reason. Peeters, Steiner and Bizer found that the best LLMs, given no or few examples, match pre-trained language models fine-tuned on thousands of labeled pairs. This snippet is the core of the Adjudicator class in er/adjudicate.py, calling the Gemini API directly with GEMINI_API_KEY.

import json
from google import genai
from google.genai import types
from pydantic import BaseModel

class Verdict(BaseModel):
    match: bool
    reason: str   # one sentence, logged for audit

PROMPT = """You are resolving product records from two catalogues.
Decide whether Record A and Record B describe the same product: same model, size, and variant.
Answer from the records only. Give a one-sentence reason.

Record A: {a}
Record B: {b}"""

client = genai.Client()   # reads GEMINI_API_KEY from the environment

def adjudicate(a_text: str, b_text: str) -> tuple[Verdict, int, int]:
    resp = client.models.generate_content(
        model="gemini-4-argon",
        contents=PROMPT.format(a=a_text, b=b_text),
        config=types.GenerateContentConfig(
            response_mime_type="application/json",
            response_schema=Verdict,
            temperature=0,
            thinking_config=types.ThinkingConfig(thinking_level="LOW"),
        ),
    )
    usage = resp.usage_metadata
    # thinking tokens are billed at the output rate, so count them with the visible answer
    out_tokens = (usage.candidates_token_count or 0) + (usage.thoughts_token_count or 0)
    return Verdict.model_validate_json(resp.text), usage.prompt_token_count or 0, out_tokens

Every verdict goes to a log with its reason, and the loop skips pairs already in the log, so a rerun never pays twice.

import os

text_of = {(src, rid): text for src, rid, text in records}

os.makedirs("results", exist_ok=True)
log_path = "results/argon_verdicts.jsonl"
done = set()
if os.path.exists(log_path):   # pairs already paid for on an earlier run
    with open(log_path) as f:
        done = {(r["abt"], r["buy"]) for r in map(json.loads, f)}

tokens_in = tokens_out = 0
with open(log_path, "a") as log:
    for abt_id, buy_id in band:
        if (abt_id, buy_id) in done:
            continue
        verdict, t_in, t_out = adjudicate(text_of["abt", abt_id], text_of["buy", buy_id])
        tokens_in += t_in
        tokens_out += t_out
        log.write(json.dumps({"abt": abt_id, "buy": buy_id, **verdict.model_dump()}) + "\n")
        log.flush()   # a crash mid-run keeps every verdict written so far

The runs in the article reached Argon through OpenRouter, which reports the exact charge per call. Thinking was set to LOW, because thinking tokens are billed and a match decision doesn't need deep reasoning. Set your key in the environment before you run the harness.

# macOS or Linux
export OPENROUTER_API_KEY="<YOUR_OPENROUTER_API_KEY>"

# Windows (PowerShell)
$env:OPENROUTER_API_KEY="<YOUR_OPENROUTER_API_KEY>"

The --budget-usdflag adds up the exact cost of every call across all runs and stops before the cap.

Argon got 960 of the 1,001 band pairs right, or 95.9%, including every hard pair the threshold had wrong. Forty of its 41 mistakes were true matches it declined.

• In 19 cases, one record was too vague to confirm, such as a microwave listed with no model number or color.
• In 7 cases, a variant, color, or bundle differed, such as a black camera against the silver one.
• In 14 cases, the model numbers differed, and several look like labeling errors in the benchmark itself.

8. Run the full pipeline

Component	Version
VectorAI DB	actian/vectorai:1.0.2
Embedding model	EmbeddingGemma (embeddinggemma) through Ollama
Adjudicator	gemini-4-argon, run on 23 October 2026 through OpenRouter, thinking level LOW
Datasets	Abt-Buy and Amazon-Google, downloaded 19 October 2026

uv run resolve.py --dataset abt-buy --argon-backend openrouter --workers 2 --budget-usd 10

Swap in --dataset amazon-google for the second benchmark, or add --no-argon to run blocking and the threshold without any API calls. Here's the trimmed output from the Abt-Buy run.

abt-buy: 1081 abt + 1092 buy records, 1097 true matches

Blocking (all records, hnsw_ef=256):
  K=1   candidates=  1,081  reduction ratio=0.9991  pair completeness=0.8186  returned/record: mean 1.0, min 1, short 0
  K=3   candidates=  3,243  reduction ratio=0.9973  pair completeness=0.9453  returned/record: mean 3.0, min 3, short 0
  K=5   candidates=  5,405  reduction ratio=0.9954  pair completeness=0.9699  returned/record: mean 5.0, min 5, short 0
  K=10  candidates= 10,810  reduction ratio=0.9908  pair completeness=0.9900  returned/record: mean 10.0, min 10, short 0
  K=20  candidates= 21,620  reduction ratio=0.9817  pair completeness=0.9973  returned/record: mean 20.0, min 20, short 0

Test split: 7,570 candidates, 767 true matches
  Global cutoff, all candidates (reference, 0.7672): P=0.5734 R=0.4785 F1=0.5217
  Best-match threshold (baseline, 0.5618): P=0.8196 R=0.8057 F1=0.8126
  Band [0.5618, 0.8405) + near-ties (rank <= 3, gap <= 0.03): 1,001 pairs (13.2% of candidates), holding 206/279 of the baseline's errors; recall ceiling 0.8970
  With Argon: P=0.9985 R=0.8449 F1=0.9153  (960/1001 band verdicts correct, $1.4081)

The results, measured

Dataset	Reduction ratio	Pair completeness	F1, threshold	F1, with Argon	Share sent to Argon	Cost per 1,000 records
Abt-Buy	99.08%	99.0%	0.813	0.915	13.2%	$0.76
Amazon-Google	99.69%	94.1%	0.597	0.691	14.0%	$0.58

On Abt-Buy, Argon rejected 135 of the 136 false matches the threshold had accepted and picked up 59 true matches that were a record's second- or third-best candidate. Amazon-Google is harder, because its software titles vary in edition, version, and packaging. Argon was right on 76.4% of that band.

The 1,001 Abt-Buy calls cost $1.41, and the 1,338 Amazon-Google calls cost $2.42. Argon's list price doubles on 1 December 2026, according to Google's Gemini API pricing page. On a CPU-only laptop with 8 GB of RAM, embedding the Abt-Buy records took 771 seconds, blocking took 35 seconds, and the Argon calls took 54 minutes with 2 requests in parallel.

Why it runs on your own infrastructure

VectorAI DB and the embedding model run on your machines, so the embeddings, the blocking, and the resolved entities stay inside your network. Only the band leaves, two records at a time. For regulated data, you have two options.

• De-identify the band by masking names, emails, and account numbers, and send only the fields that decide the match.
• Keep the second pass local by swapping adjudicate() for a call to a model you host. A local model wasn't measured, so expect different numbers.

When the Argon pass is worth it

Run it when the band holds most of the threshold's errors and Argon decides band pairs at least 10 points more accurately than the threshold.

Dataset	Threshold's errors in the band	Threshold right on band pairs	Argon right on band pairs	F1 gain
Abt-Buy	206 of 279 (74%)	79.4%	95.9%	+10.3
Amazon-Google	500 of 697 (72%)	62.6%	76.4%	+9.4

Doubling the band's width on Abt-Buy sent 78 more pairs to Argon for $0.12 more, and F1 moved from 0.9153 to 0.9160. To check your own data, label about 200 band pairs by hand and compare how often the threshold and Argon get them right. That sample costs about 30 cents in Argon calls.

Wrapping up

The second pass got the Sharp TVs right. The threshold picked the LC37D44U because its vector scored higher, while Argon matched the LC37D64U by its model number and logged why. Blocking removed 99% of the comparison space, data-driven band edges kept the paid work to 13.2% of candidates, and Argon read the part numbers the embedding blurred.

What's next

• Point the loader at your own customer, supplier, or product records and put your exclusion rules in the blocking filter's must clause.
• Run it on VectorAI DB Community Edition.
• Add vector search to an existing service with VectorAI DB and FastAPI.

docker pull actian/vectorai:latest