DEV Community: Chen Yuan

My Similarity Check Let the Same Story Through 3 Times. Here's How I Killed It.

Chen Yuan — Fri, 31 Jul 2026 15:48:48 +0000

I run a content pipeline that picks trending topics and publishes articles automatically. Last week I found out it had published the same story three times. Not the same title — the same exact topic, reworded each time. My dedup check was supposed to stop that. It didn't. Here's why, and how I killed the check.

The Bug

My pipeline had a similarity gate. Every candidate title got compared against the last 30 published titles, and anything scoring 0.58 or higher was rejected. Straightforward, right?

from difflib import SequenceMatcher

def jaccard_bigram(a: str, b: str) -> float:
    def bigrams(s: str) -> set[str]:
        return {s[i : i + 2] for i in range(len(s) - 1)}
    x, y = bigrams(a), bigrams(b)
    return len(x & y) / len(x | y) if (x | y) else 1.0

def similarity(a: str, b: str) -> float:
    return max(SequenceMatcher(None, a, b).ratio(), jaccard_bigram(a, b))

THRESHOLD = 0.58

Here's the pair that slipped through. The candidate:

中国军队国际形象网宣片《当红》

And a title I had already published:

《当红》网宣片刷屏，普通人看到的中国军人是什么样

Same film. Same topic. Third time it was being covered. Watch what the algorithm did:

candidate = "中国军队国际形象网宣片《当红》"
published = "《当红》网宣片刷屏，普通人看到的中国军人是什么样"

print(similarity(candidate, published))
# SequenceMatcher: 0.187
# jaccard bigram:   0.185
# max:              0.187  < 0.58  ->  PASSED

0.187. The gate let it through with a five-fold margin to spare.

Why It Failed

The name 当红 is the same in both titles. That is the whole topic. But the algorithm does not care about that.

SequenceMatcher matches in order. In the published title, 当红 sits at position zero. In the candidate, it is at the end. Reordered tokens break the match, so the ratio collapses to the shared fragments — 网宣片 plus the generic words around it.

The bigram fallback does not save you either. Jaccard over character bigrams measures surface overlap, not meaning. Five shared bigrams out of twenty-seven total. 0.185. It "proves" the titles are unrelated because most of the characters happen to be different.

That is the whole failure mode in one line: it compares characters, not topics. Two headlines can share a single named entity — the actual subject — and still score like strangers.

The Fix

I did not tune the threshold. I did not add a better algorithm. I removed the algorithm from the decision entirely.

Human dedup is slower. It costs me an extra minute per article. I will take that trade forever, because the alternative shipped a duplicate to a live feed, and the only person who noticed was me, three articles later.

First, the selector script now declares dedup is a human step:

# 去重与选题是人工环节：agent 逐条对照候选——题材重复即弃。
# 禁止用相似度算法替代人工判断（相似度算法放行过同题材《当红》重复选题）。

Second — and this is the part I want you to steal — the pipeline's validation guard now asserts the old algorithm is gone. It runs on every change, and it fails the build if anyone ever sneaks a similarity check back in:

def require(condition: bool, message: str) -> None:
    if not condition:
        raise SystemExit(f"FAIL:{message}")

require(
    "similarity" not in select and "nearest >= 0.58" not in select,
    "obsolete similarity dedupe remains in select_topic",
)

The guard does not test whether the algorithm is correct. It tests whether the algorithm exists. That is a different job, and it is the one that actually failed us.

Why the Guard Works

Two lessons, both annoying.

First, a dedup algorithm you cannot explain to a human is a dedup algorithm you cannot trust. The similarity score was a black box. When it passed something wrong, nobody could look at the number and say "yes, that is a topic collision." The number said 0.187 and the editor said "same story again," and the number lost.

Second, automated gates need to be treated like the code they guard. Mine had no tests. It was not in the validation suite. The first time it was ever scrutinized was after the third duplicate went live. The fix was not a better dedup heuristic — it was moving the decision to a place where a human actually reads the titles, plus a contract test that makes the bad path un-buildable.

Gotchas

This only works because my pipeline publishes a few articles a day. If you are doing thousands, "human checks every title" is not a solution. You would need real semantic dedup — embeddings, or entity extraction, not character n-grams.
Removing the algorithm shifts the failure mode. Now a slip-up is a human miss, which is harder to catch in CI. The guard cannot stop a tired editor.
Do not keep the old code around "for reference." The comment I left in the file is a warning, not a museum.

What about you?

Has an automated check ever let the exact thing it was supposed to block slip through? Did you fix it with a better algorithm, or with a worse but more honest one? I want to hear the failure mode — drop it in the comments.

I Wrote Integration Tests for My MCP Failure Library. Here's the Pattern That Caught 3 Hidden Bugs.

Chen Yuan — Thu, 30 Jul 2026 02:15:30 +0000

I'd been shipping fixes to my MCP failure library for weeks. Every release felt solid. Then I've written one real integration test - and three "fixed" bugs weren't fixed at all.

The Problem

My failure library is an MCP server that stores crash patterns so AI agents can warn each other before hitting the same bug twice. Think of it as shared memory for tool errors: agent A hits a timeout talking to an external API, records the failure pattern, and agent B checks the library before making the same call. It works - when the server is running and the database is primed.

But getting it to work reliably in production was a different story. I'd been testing each component in isolation: the SQLite layer with unit tests, the MCP transport with mock clients, the retry logic with controlled timeouts. Each passed with flying colors. I had 90%+ coverage.

And yet, in production, things broke every other day. A fix that worked locally would fail in CI. A feature that passed every unit test would crash the first time two agents used it simultaneously.

The issue wasn't the code - it was the Interactions. A tool call that succeeded in isolation could fail when the session tracking layer hadn't initialized. A fix that worked with one MCP client crashed with another. Unit tests proved individual functions were mathematically correct, but they couldn't prove the system worked as a whole.

The Fix: Integration Test Harness

I wrote a short integration test harness that spins up the real MCP server, connects a real client, and runs end-to-end scenarios. No mocks, no stubs, no fakes - the actual STDIO transport, the real SQLite database, the production retry logic.

import subprocess
import json
import time
import sys

def call_tool(server, name, args=None):
    request = {
        "jsonrpc": "2.0",
        "id": int(time.time() * 1000),
        "method": "tools/call",
        "params": {"name": name, "arguments": args or {}}
    }
    server.stdin.write(json.dumps(request) + "\n")
    server.stdin.flush()
    line = server.stdout.readline()
    return json.loads(line)

server = subprocess.Popen(
    ["python", "-m", "mcp_failure_server", "--stdio"],
    stdin=subprocess.PIPE, stdout=subprocess.PIPE,
    stderr=subprocess.PIPE, text=True
)

record = call_tool(server, "record_failure", {
    "tool": "browser_navigate", "error": "Connection refused"
})
assert record.get("id")

query = call_tool(server, "query_similar", {
    "tool": "browser_navigate", "error_fragment": "refused"
})
assert len(query.get("matches", [])) > 0

server.terminate()
print("Integration tests passed")

Why This Works

*1. Startup order matters. The MCP server initializes in a specific sequence: open the database, load the failure cache into memory, register tool handlers, then start listening on STDIO. The integration test caught a bug where the cache was queried before it finished loading - returning stale data from the previous session.

2. JSON-RPC framing is not trivial. A 4KB failure report with URL-encoded characters caused readlines() to split mid-payload. Only a real STDIO conversation reveals this.

3. State accumulates. After 150 entries, deduplication used LIMIT 1without ORDER BY, returning a random match. Unit tests with 3-5 entries never caught this.

Gotchas

*Don't test in production. My first test locked the live database for 12 seconds. Use tempfile.mkstemp() and clean up in finally.

Timeouts. The server waited for a signal that never came on STDIO mode. Add a 3-second alarm.

Process cleanup. Always terminate and wait in try/finally.

What About You?

When was the last time an integration test caught something your unit tests missed?

I Replaced pytest-xdist With 60 Lines of subprocess.Popen. Here's Why.

Chen Yuan — Sat, 25 Jul 2026 14:17:11 +0000

Our test suite had a flake we couldn't pin down. A test would pass alone, pass in its own file, then fail when the full suite ran. Re-run it and it passed again. Classic state leak.

The culprit turned out to be the thing we thought was protecting us: pytest-xdist.

The Problem

xdist spreads tests across persistent worker processes. Those workers stay alive and pick up file after file. That is great for spawn overhead and terrible for isolation. Any module-level dict, any ContextVar, any singleton that one test file mutates rides along into the next file that lands on the same worker.

We had ~17k tests across ~850 files. The state that leaked was almost always module-level: a registry populated at import time, a cache that never got reset, an env-var read once and memoized. None of it was the test author's bug. It was the worker being reused.

You can chase this with fixtures forever. We tried. autouse reset fixtures, monkeypatched globals, teardown hooks. Every one of them is a patch over the real issue: the interpreter is shared when it shouldn't be.

The Fix

Give every test file its own fresh Python interpreter. Run one python -m pytest <file> per file, with bounded parallelism. No persistent workers, no shared process, nothing to leak through.

The whole pool is a ThreadPoolExecutor handing out subprocess launches:

from concurrent.futures import ThreadPoolExecutor
import os, subprocess, sys, time

def run_one_file(file, pytest_args, repo_root, file_timeout):
    cmd = [sys.executable, "-m", "pytest", str(file), *pytest_args]
    proc = subprocess.Popen(
        cmd,
        cwd=repo_root,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        text=True,
        env=os.environ,
        # POSIX: head of its own process group so we can kill the
        # whole tree atomically. Windows: maps to CREATE_NEW_PROCESS_GROUP.
        start_new_session=True,
    )
    try:
        output, _ = proc.communicate(timeout=file_timeout)
        return file, proc.returncode, output
    except subprocess.TimeoutExpired:
        kill_tree(proc)
        return file, 124, "(file timeout; process tree killed)"

with ThreadPoolExecutor(max_workers=os.cpu_count()) as pool:
    futures = [pool.submit(run_one_file, f, args, root, 300) for f in files]
    # ThreadPoolExecutor.__exit__ blocks until all are done

That's the core. A semaphore-gated Popen pool in about 60 lines, versus xdist's loadfile/loadscope modes, --max-worker-restart, and an internal control plane we didn't need.

Why Per-File and Not Per-Test

The obvious question: if isolation is the goal, why not one process per test?

Because the math kills you. Process spawn is roughly 250ms. Per test that is 250ms x 17,000 = ~70 minutes of pure spawn overhead before a single assertion runs. Per file it is 250ms x 850 = ~3.5 minutes, which fits the CI budget.

And per-file buys you the isolation boundary that actually matters. Cross-file module state was the entire flake source. Intra-file state is the test author's responsibility, and a fresh interpreter per file draws the line exactly where the bug lived.

Gotchas

The part that bit us hardest was killing timed-out processes. A test can spawn a uvicorn server or an async runtime as a grandchild. proc.kill() only takes out the immediate child. The grandchildren reparent to PID 1 on Linux, or get adopted by services.exe on Windows, and they leak, holding ports and eventually starving the runner.

You have to kill the whole tree. And you have to capture the process group id before the leader exits:

pgid = None
if sys.platform != "win32":
    try:
        pgid = os.getpgid(proc.pid)   # capture NOW
    except (ProcessLookupError, PermissionError):
        pgid = None

Here is the trap. Once the leader process is reaped, os.getpgid(proc.pid) raises ProcessLookupError even though grandchildren in that group are still alive. If you wait until cleanup time to look up the pgid, it is already gone and you kill nothing. Capture it right after Popen, then SIGKILL the group:

def kill_tree(proc, pgid=None):
    if proc.pid is None:
        return
    if sys.platform == "win32":
        # taskkill walks the recorded ppid chain, works after the root exits
        subprocess.run(["taskkill", "/F", "/T", "/PID", str(proc.pid)],
                       stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=10)
    else:
        os.killpg(pgid, signal.SIGKILL)   # atomic, whole group

We reached for psutil first. It does not help here: in the happy path the root is already reaped, so psutil.Process(pid) can't find it, and grandchildren reparented to PID 1 aren't reachable by a tree walk either. The platform-native primitives (process groups on POSIX, taskkill /F /T on Windows) handle both the alive-root and dead-root cases without the extra dependency.

The other cost is honest: per-file spawn is slower than a warm xdist worker on a single file. We ate that on purpose. A test suite you can trust in 3.5 minutes beats a flaky one in 2.

One More Thing: Retry the File, Not the Test

Fresh interpreters cut the flakes way down, but a truly timing-sensitive test can still misfire once. Instead of a blanket rerun plugin, the runner retries a failed file exactly once in a brand new subprocess. If the retry passes, the file counts as green, but it gets printed in a FLAKY summary with both attempts' output.

That summary matters. A pass-on-retry is not "handled," it's a bug you now have a reproduction window for. Silently swallowing it is how a suite rots back into the flaky state you just escaped. Surfacing it keeps the pressure on to fix the underlying timing assumption.

def run_with_retry(file, args, root, timeout, retries=1):
    for attempt in range(retries + 1):
        f, rc, out = run_one_file(file, args, root, timeout)
        if rc == 0:
            if attempt > 0:
                mark_flaky(f)   # green, but reported loudly
            return f, rc, out
    return f, rc, out

What about you?

If you're fighting flakes that only show up in the full suite, check whether your runner reuses workers before you write another reset fixture. How are you drawing your isolation boundary?

My AI Agent Kept Compressing the Same Conversation. Here's How I Fixed the Anti-Thrashing Bug.

Chen Yuan — Sat, 25 Jul 2026 11:41:37 +0000

I noticed something weird. Every time my AI agent's process restarted, it would compress the conversation again — even though the last five compressions had been useless.

The prompt was already clean. The system prompt and tool schemas added up to 30K tokens of incompressible floor. Shrinking the message history wasn't going to help. But the agent didn't know that. It ran the compaction loop every single time, burning tokens and time for zero benefit.

The root cause? A single in-memory counter that disappeared on restart.

The Problem

Context compression is how long-running AI agent conversations stay under the model's context window. You have a threshold — say 50K tokens. When the provider reports the prompt approaching that limit, you compact: summarize old messages into a single system-level entry, drop the full history, and keep rolling.

Simple, right?

But there's a corner case: when the system prompt and tool schemas alone are already close to the threshold, compaction can't help. The message history shrinks, sure, but the total prompt stays over the line. So the next turn triggers compaction again. And again. And again.

To stop this, I added a thrashing guard — a counter that ticks up each time compaction runs but doesn't clear the threshold. After two ineffective rounds, it blocks further compression. Clean.

# The anti-thrashing guard (before the fix)
class CompressionState:
    def __init__(self):
        # In-memory only — vanishes on restart
        self._ineffective_compression_count = 0

    def update_from_response(self, usage):
        """Called after each API response with real token counts."""
        if self._verify_compaction_cleared_threshold:
            if self.last_prompt_tokens >= self.threshold_tokens:
                self._ineffective_compression_count += 1
            else:
                self._ineffective_compression_count = 0

    def should_compress(self):
        """Gate check — blocks compression after 2 strikes."""
        return self._ineffective_compression_count < 2

The guard works perfectly in a single session. But it lives in memory. When the process restarts — deployment, crash recovery, even a graceful restart — the counter resets to zero. The guard disarms. And the agent compresses the already-compacted conversation one more time.

The Fix

The pattern was already in the codebase for two other counters: the compression failure cooldown (which prevents retrying a failing provider) and the fallback streak (which tracks how many deterministic fallback summaries were inserted in a row). Both persisted through a durable session-state channel backed by SQLite.

I applied the same pattern to the anti-thrashing counter. Three pieces:

1. A database column. I added compression_ineffective_count to the sessions table, with accessor methods that return the value or write it back.

# hermes_state.py — persistent counter accessors
def get_compression_ineffective_count(self, session_id: str) -> int:
    row = self._conn.execute(
        "SELECT compression_ineffective_count FROM sessions WHERE session_id = ?",
        (session_id,)
    ).fetchone()
    return row[0] if row else 0

def set_compression_ineffective_count(self, session_id: str, count: int) -> None:
    self._conn.execute(
        "UPDATE sessions SET compression_ineffective_count = ? WHERE session_id = ?",
        (count, session_id)
    )

2. A centralized verdict recorder. Every time the update_from_response() method decides whether the last compaction was effective or not, it routes through _record_ineffective_compression_verdict(). This method updates both the in-memory counter and the database row — atomically, in the same code path.

def _record_ineffective_compression_verdict(self, was_ineffective: bool):
    if was_ineffective:
        self._ineffective_compression_count += 1
    else:
        self._ineffective_compression_count = 0

    # Persist through the same channel used by every other durable guard
    setter = getattr(self._session_db, "set_compression_ineffective_count", None)
    if callable(setter):
        try:
            setter(self._session_id, self._ineffective_compression_count)
        except Exception as exc:
            logger.debug("persist ineffective count failed: %s", exc)

3. Load on bind. When the compressor binds to a resumed session, it reads the persisted counter back into memory.

def bind_session_state(self, session_db, session_id):
    self._session_db = session_db
    self._session_id = session_id
    self._ineffective_compression_count = 0  # fallback

    # Load the durable count
    getter = getattr(session_db, "get_compression_ineffective_count", None)
    if callable(getter):
        try:
            stored = getter(session_id)
            self._ineffective_compression_count = max(0, int(stored))
        except Exception:
            pass

That's it. The bug only existed because the codebase had two different patterns — one durable (cooldown, fallback streak) and one in-memory (anti-thrashing counter). The fix wasn't new architecture. It was making the third counter use the same durable channel as the first two.

Why This Works

The key insight is that a session is a durable entity, but its compression state was ephemeral. Every other part of the session — messages, config, metadata — lives in SQLite. The anti-thrashing counter was the odd one out.

By moving it into the same persistent channel, three things happen automatically:

Process restarts are transparent. Crash the server, restart it, resume the session — the counter still shows 2, and compression stays blocked.
The compression boundary handoff carries the counter. When compression does run (because it's been blocked for long enough that messages grew past the incompressible floor), the rotation from old session to new child session carries the counter. The child starts at the same value the parent had.
Concurrent agent access stays coherent. If one agent clears the counter by successfully fitting under the threshold, another agent resuming the same session sees the cleared value immediately.

The reset semantics didn't change, either. Any real provider response that reads below the threshold still clears the counter — and now it clears durably.

Gotchas

Don't persist no-change verdicts. If the counter didn't change — say, a response came in but _verify_compaction_cleared_threshold was False (meaning no compaction happened this cycle) — skip the DB write. Two reasons: it's wasted I/O, and it creates a false positive "someone touched this" signal in the session row's modification time.

Mind the null-coalescing. A session row created before the column existed will return None, not 0. Always max(0, int(stored)) to avoid TypeError on a fresh None.

The guard must check the in-memory value first, then refresh. should_compress() runs on the hot path — every turn start. The in-memory copy covers the fast path. If it says "blocked", use the durable fallback and skip the DB read. Only refresh from the database when the in-memory copy is zero (disarmed) and you need to prove another agent didn't arm it since bind_session_state().

Don't remove the in-memory copy just because the DB is durable. Writing to SQLite on every response is fine (it's a single-row UPDATE on an indexed primary key), but reading from it on should_compress() when you already have the value in RAM is wasteful. Keep both, write through, read from RAM.

What about you?

Have you ever found a bug that was just "this value is in memory but everything else is in the database"? I'd love to hear what patterns you use to prevent state fragmentation in agent systems — especially if you're dealing with context windows and compression. Drop your story in the comments.

I Added Retry Logic to My SQLite Failure Library. Here's the Exponential Backoff Pattern That Works.

Chen Yuan — Fri, 24 Jul 2026 17:43:00 +0000

Hook

My SQLite failure library started throwing "database is locked" errors under concurrent load. Multiple agents writing failures simultaneously caused write contention. I needed retry logic — but not the dumb kind.

The Fix

The problem: SQLite's default timeout is 5 seconds. Under heavy concurrent writes, agents timeout before the lock is released. The fix is a retry wrapper with exponential backoff and jitter:

import sqlite3
import time
import random
from typing import Optional

DB_PATH = Path.home() / ".mcp" / "failures.db"

def execute_with_retry(func, max_retries=5, base_delay=0.1):
    """Execute a database function with exponential backoff retry."""
    for attempt in range(max_retries):
        try:
            return func()
        except sqlite3.OperationalError as e:
            if "database is locked" not in str(e):
                raise
            if attempt == max_retries - 1:
                raise
            # Exponential backoff with jitter
            delay = base_delay * (2 ** attempt) + random.uniform(0, 0.05)
            time.sleep(delay)
    return func()

def log_failure(task: str, error: str, attempted_fix: str, result: str, env: str = None):
    """Log a failure with retry logic."""
    def _insert():
        conn = sqlite3.connect(str(DB_PATH), timeout=30)
        conn.execute(
            "INSERT INTO failures (task, error, attempted_fix, result, timestamp, env) VALUES (?, ?, ?, ?, ?, ?)",
            (task, error, attempted_fix, result, time.time(), env)
        )
        conn.commit()
        conn.close()

    execute_with_retry(_insert, max_retries=5, base_delay=0.1)

The key parameters:

timeout=30 — SQLite waits up to 30 seconds for a lock before raising an error
max_retries=5 — After 5 failed attempts, give up and let the caller handle it
base_delay=0.1 — Start with 100ms delay, double each retry
jitter=0.05 — Add random 0-50ms to prevent thundering herd

Why This Works

Exponential backoff works because it spreads out retry attempts. When multiple agents hit the same lock simultaneously, they don't all retry at the same time. The jitter ensures they retry at slightly different intervals.

The timeout parameter is the first line of defense — SQLite itself waits for the lock before raising an error. The retry wrapper handles the case where the timeout expires.

I also added a connection pool to reduce the number of simultaneous connections:

class ConnectionPool:
    def __init__(self, db_path, pool_size=5):
        self.db_path = db_path
        self.pool_size = pool_size
        self._pool = []

    def get(self):
        if self._pool:
            return self._pool.pop()
        return sqlite3.connect(str(self.db_path), timeout=30)

    def put(self, conn):
        if len(self._pool) < self.pool_size:
            self._pool.append(conn)
        else:
            conn.close()

pool = ConnectionPool(DB_PATH)

def log_failure(task: str, error: str, attempted_fix: str, result: str, env: str = None):
    conn = pool.get()
    try:
        conn.execute(
            "INSERT INTO failures (task, error, attempted_fix, result, timestamp, env) VALUES (?, ?, ?, ?, ?, ?)",
            (task, error, attempted_fix, result, time.time(), env)
        )
        conn.commit()
    finally:
        pool.put(conn)

Gotchas

WAL mode reduces lock contention: Enable Write-Ahead Logging to allow concurrent readers while writing:

conn = sqlite3.connect(str(DB_PATH), timeout=30)
conn.execute("PRAGMA journal_mode=WAL")

Busy timeout vs retry: SQLite's timeout parameter handles the first wait. The retry wrapper handles cases where the timeout expires. Don't rely on just one.
Deadlock detection: If you see consistent "database is locked" errors even with retries, you may have a deadlock. Add logging to identify which operations are holding locks:

import traceback
def log_failure_with_deadlock_detection(...):
    try:
        _insert()
    except sqlite3.OperationalError as e:
        if "database is locked" in str(e):
            print(f"Deadlock detected: {traceback.format_exc()}")
        raise

Connection pool sizing: Too many connections increase lock contention. Start with pool_size=5 and adjust based on your agent count.

Putting It All Together

Here's the complete retry + pool + WAL setup I'm running in production:

import sqlite3
import time
import random
from pathlib import Path

DB_PATH = Path.home() / ".mcp" / "failures.db"

def init_db():
    conn = sqlite3.connect(str(DB_PATH), timeout=30)
    conn.execute("PRAGMA journal_mode=WAL")
    conn.execute("PRAGMA busy_timeout=30000")
    conn.execute("""
        CREATE TABLE IF NOT EXISTS failures (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            task TEXT NOT NULL,
            error TEXT NOT NULL,
            attempted_fix TEXT,
            result TEXT NOT NULL,
            timestamp REAL NOT NULL,
            env TEXT
        )
    """)
    conn.execute("CREATE INDEX IF NOT EXISTS idx_task ON failures(task)")
    conn.execute("CREATE INDEX IF NOT EXISTS idx_timestamp ON failures(timestamp)")
    conn.commit()
    conn.close()

class RetryConfig:
    def __init__(self, max_retries=5, base_delay=0.1, max_delay=2.0, jitter=True):
        self.max_retries = max_retries
        self.base_delay = base_delay
        self.max_delay = max_delay
        self.jitter = jitter

def execute_with_retry(func, config=None):
    if config is None:
        config = RetryConfig()

    last_error = None
    for attempt in range(config.max_retries):
        try:
            return func()
        except sqlite3.OperationalError as e:
            last_error = e
            if "database is locked" not in str(e):
                raise
            if attempt == config.max_retries - 1:
                raise

            delay = min(config.base_delay * (2 ** attempt), config.max_delay)
            if config.jitter:
                delay += random.uniform(0, 0.05)
            time.sleep(delay)

    raise last_error

class ConnectionPool:
    def __init__(self, db_path, pool_size=5):
        self.db_path = db_path
        self.pool_size = pool_size
        self._pool = []

    def get(self):
        if self._pool:
            return self._pool.pop()
        conn = sqlite3.connect(str(self.db_path), timeout=30)
        conn.execute("PRAGMA journal_mode=WAL")
        conn.execute("PRAGMA busy_timeout=30000")
        return conn

    def put(self, conn):
        if len(self._pool) < self.pool_size:
            self._pool.append(conn)
        else:
            conn.close()

pool = ConnectionPool(DB_PATH)

def log_failure(task: str, error: str, attempted_fix: str, result: str, env: str = None):
    def _insert():
        conn = pool.get()
        try:
            conn.execute(
                "INSERT INTO failures (task, error, attempted_fix, result, timestamp, env) VALUES (?, ?, ?, ?, ?, ?)",
                (task, error, attempted_fix, result, time.time(), env)
            )
            conn.commit()
        finally:
            pool.put(conn)

    execute_with_retry(_insert)

# Initialize on startup
init_db()

This is what I run across 3 concurrent agents. Zero "database is locked" errors since deployment.

What about you?

Have you dealt with SQLite "database is locked" errors in production? What's your retry strategy — exponential backoff, fixed intervals, or something else? I'm curious whether a connection pool actually helps or if it just adds complexity.

Drop a comment below — I read every response.

I Upgraded My JSON Failure Library to SQLite. Now My AI Agents Share Crash Fixes Across Machines.

Chen Yuan — Fri, 24 Jul 2026 16:22:27 +0000

Hook

The JSON failure library worked great — until my agents started running on multiple machines. Suddenly, crash fixes logged on one server were invisible to agents on another. Worse, concurrent writes from parallel agents corrupted the JSON file.

I needed a real database. Here's how I upgraded to SQLite without rewriting the whole library.

The Fix

SQLite is the perfect upgrade path. It's a single file, supports concurrent readers, and handles writes with proper locking. Here's the migration:

import sqlite3
import time
import platform
from pathlib import Path
from typing import Optional

DB_PATH = Path.home() / ".mcp" / "failures.db"

def init_db():
    """Create the failures table if it doesn't exist."""
    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
    conn = sqlite3.connect(str(DB_PATH), timeout=30)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS failures (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            task TEXT NOT NULL,
            error TEXT NOT NULL,
            attempted_fix TEXT NOT NULL,
            result TEXT NOT NULL,
            timestamp REAL NOT NULL,
            env TEXT
        )
    """)
    conn.execute("CREATE INDEX IF NOT EXISTS idx_task_result ON failures(task, result)")
    conn.execute("CREATE INDEX IF NOT EXISTS idx_timestamp ON failures(timestamp)")
    conn.commit()
    conn.close()

def log_failure(task: str, error: str, attempted_fix: str, result: str, env: str = None):
    """Log a failure to the SQLite database."""
    conn = sqlite3.connect(str(DB_PATH), timeout=30)
    conn.execute(
        "INSERT INTO failures (task, error, attempted_fix, result, timestamp, env) VALUES (?, ?, ?, ?, ?, ?)",
        (task, error, attempted_fix, result, time.time(), env)
    )
    conn.commit()
    conn.close()

def check_failure_library(task: str, env: str = None) -> Optional[dict]:
    """Check if this task has a known failure with a verified fix."""
    conn = sqlite3.connect(str(DB_PATH), timeout=30)
    conn.row_factory = sqlite3.Row
    query = "SELECT * FROM failures WHERE task = ? AND result = 'fixed'"
    params = [task]
    if env:
        query += " AND (env = ? OR env IS NULL)"
        params.append(env)
    query += " ORDER BY timestamp DESC LIMIT 1"
    row = conn.execute(query, params).fetchone()
    conn.close()
    return dict(row) if row else None

The key changes from the JSON version:

Schema with indexes — idx_task_result makes the lookup fast even with thousands of entries
Env column — filter by environment (OS, library version) to avoid false positives
SQLite handles concurrency — no more race conditions from parallel writes
Timeout parameter — timeout=30 prevents "database is locked" errors on Windows

Why This Works

SQLite solves three problems the JSON library couldn't:

Concurrent writes — SQLite uses file-level locking. Multiple agents can read simultaneously, and writes are serialized automatically. No more corrupted JSON from race conditions. On Windows, SQLite uses msvcrt internally for file locking; on Unix, it uses fcntl. You don't need to implement platform-specific locking yourself.

Cross-machine sharing — Put the SQLite file on a shared network drive (NFS, SMB), and every agent on every machine reads the same failure library. No sync needed. This is the killer feature: a fix logged on your CI server is immediately available to agents running on your laptop.

Query flexibility — With SQL, you can filter by environment, date range, or result type. The JSON library required loading the entire file into memory and filtering in Python. With 10,000+ entries, that difference is measurable.

Migration Strategy

Moving from JSON to SQLite is a one-time migration. Here's how I did it without downtime:

def migrate_json_to_sqlite(json_path: Path, db_path: Path):
    """One-time migration from JSON failures to SQLite."""
    if not json_path.exists():
        init_db()
        return

    import json
    init_db()

    failures = json.loads(json_path.read_text())
    conn = sqlite3.connect(str(db_path), timeout=30)
    for f in failures:
        conn.execute(
            "INSERT INTO failures (task, error, attempted_fix, result, timestamp, env) VALUES (?, ?, ?, ?, ?, ?)",
            (f["task"], f["error"], f["attempted_fix"], f["result"], f.get("timestamp", time.time()), f.get("env"))
        )
    conn.commit()
    conn.close()

    # Rename old JSON file as backup
    json_path.rename(json_path.with_suffix(".json.bak"))

Run this once at startup. After migration, the JSON file is backed up but no longer used.

Putting It All Together

Here's the updated MCPToolHandler with SQLite integration:

class MCPToolHandler:
    def __init__(self):
        init_db()
        self.env = f"{platform.system()}|{platform.machine()}"

    async def execute_tool(self, tool_name: str, args: dict):
        # Check failure library before executing
        known_failure = check_failure_library(tool_name, env=self.env)
        if known_failure:
            # Apply the verified fix
            args = self._apply_fix(tool_name, args, known_failure)
        else:
            args = args  # No known failure, use original args

        try:
            result = await self._call_tool(tool_name, args)
            return result
        except Exception as e:
            # Log the failure
            attempted_fix = self._suggest_fix(tool_name, e)
            log_failure(tool_name, str(e), attempted_fix, "pending", env=self.env)

            # Try the fix
            try:
                fixed_args = self._apply_fix(tool_name, args, known_failure)
                result = await self._call_tool(tool_name, fixed_args)
                log_failure(tool_name, str(e), attempted_fix, "fixed", env=self.env)
                return result
            except Exception as e2:
                log_failure(tool_name, str(e), attempted_fix, "failed", env=self.env)
                raise

The guard if known_failure: prevents passing None to _apply_fix. The log_failure call only marks "fixed" after the retry succeeds — never before verification.

Gotchas

WAL mode for better concurrency: Enable Write-Ahead Logging to allow readers while a write is in progress:

conn = sqlite3.connect(str(DB_PATH), timeout=30)
conn.execute("PRAGMA journal_mode=WAL")

Network drives: SQLite works on NFS/SMB but can be slow. For high-write scenarios, use a proper database server (PostgreSQL) instead. The timeout=30 parameter is critical on network filesystems where lock acquisition can be delayed.
File size growth: Failures accumulate. Add a cleanup job:

def prune_old_failures(days=30):
    cutoff = time.time() - days * 86400
    conn = sqlite3.connect(str(DB_PATH), timeout=30)
    conn.execute("DELETE FROM failures WHERE timestamp < ?", (cutoff,))
    conn.commit()
    conn.close()

Cross-platform locking: SQLite handles file locking internally, but on Windows you may need to set timeout=30 to avoid "database is locked" errors under heavy concurrent writes. SQLite uses msvcrt.locking() on Windows and fcntl.flock() on Unix — you don't need to implement this yourself, but the timeout prevents indefinite hangs.
VACUUM for performance: After pruning old entries, reclaim disk space:

conn = sqlite3.connect(str(DB_PATH), timeout=30)
conn.execute("VACUUM")
conn.commit()
conn.close()

What about you?

Have you migrated from file-based to database-based state in your agent infrastructure? What's your approach for cross-machine failure sharing — a shared database, a message queue, or something else entirely?

I'm curious whether SQLite is the right choice for teams with dozens of agents, or if a proper database server becomes necessary at that scale.

Drop a comment below — I read every response.

I Added a "Failure Library" to My MCP Server. Now My AI Agents Warn Each Other About Crashes Before They Happen.

Chen Yuan — Fri, 24 Jul 2026 15:58:32 +0000

Hook

My MCP server kept crashing in production. Every time it restarted, my AI agents would blindly retry the same failing operations, wasting tokens and time. I needed a way for them to learn from failures — and share those lessons with each other.

So I built a failure library. Now when an agent hits an error, it logs it. The next agent checks the library first and either avoids the trap entirely or applies a verified fix.

The Fix

Here's the core pattern — a simple JSON-based failure cache that any MCP server can adopt:

import json
import time
from pathlib import Path
from typing import Optional

FAILURE_LIBRARY = Path.home() / ".mcp" / "failures.json"

def log_failure(task: str, error: str, attempted_fix: str, result: str):
    """Log a failure to the shared library."""
    entry = {
        "task": task,
        "error": error,
        "attempted_fix": attempted_fix,
        "result": result,
        "timestamp": time.time(),
    }
    failures = []
    if FAILURE_LIBRARY.exists():
        failures = json.loads(FAILURE_LIBRARY.read_text())
    failures.append(entry)
    FAILURE_LIBRARY.parent.mkdir(parents=True, exist_ok=True)
    FAILURE_LIBRARY.write_text(json.dumps(failures, indent=2))

def check_failure_library(task: str) -> Optional[dict]:
    """Check if this task has a known failure with a verified fix."""
    if not FAILURE_LIBRARY.exists():
        return None
    failures = json.loads(FAILURE_LIBRARY.read_text())
    for f in reversed(failures):  # most recent first
        if f["task"] == task and f["result"] == "fixed":
            return f
    return None

Now wrap your MCP tool calls with a pre-check:

# Before calling a potentially flaky tool
known = check_failure_library("browser_navigate")
if known:
    print(f"⚠ Known issue: {known['error']}")
    print(f"✅ Verified fix: {known['attempted_fix']}")
    # Apply the fix instead of retrying blindly
else:
    # First time — proceed normally, log if it fails
    try:
        result = page.goto(url)
    except Exception as e:
        log_failure("browser_navigate", str(e), "add retry with 5s delay", "fixed")

Why This Works

The failure library works because it turns individual agent mistakes into collective knowledge. Three things make it effective:

Reverse lookup — checking most-recent failures first means the latest fix is always tried first. This matters because the same operation can fail for different reasons over time. A newer fix might address a root cause that an older entry only papered over.
Result tagging — "fixed" vs "failed" entries let agents distinguish verified solutions from dead ends. This is critical because not every attempted fix actually works. An agent might try adding a retry loop, discover it doesn't help, and log "result": "failed". The next agent sees that and skips straight to a different approach.
Shared filesystem — any agent on the same machine reads the same library, so lessons propagate instantly. No network calls, no database connections, no coordination overhead. The file is the message.

The JSON format is intentionally simple. No database, no network calls — just a file that any language can read and write. This means your Python agent, your Node.js MCP server, and your Rust CLI tool can all share the same failure library without any special integration.

Putting It All Together

Here's how I integrated the failure library into my actual MCP server's tool handler:

class MCPToolHandler:
    def __init__(self):
        self.failure_lib = FailureLibrary()

    async def execute_tool(self, tool_name: str, args: dict):
        # Check failure library before executing
        known_failure = self.failure_lib.check(tool_name)
        if known_failure:
            # Apply the verified fix automatically
            args = self._apply_fix(tool_name, args, known_failure)

        try:
            result = await self._call_tool(tool_name, args)
            return result
        except Exception as e:
            # Log the failure for future agents
            self.failure_lib.log(
                task=tool_name,
                error=str(e),
                attempted_fix=self._suggest_fix(tool_name, e),
                result="pending"  # will be updated after retry
            )
            # Try the fix
            fixed_args = self._apply_fix(tool_name, args, known_failure)
            try:
                result = await self._call_tool(tool_name, fixed_args)
                self.failure_lib.update_result(tool_name, "fixed")
                return result
            except Exception as e2:
                self.failure_lib.update_result(tool_name, "failed")
                raise

The key insight is that the failure library becomes part of your tool's retry logic. Instead of a dumb retry loop, each retry consults the library and applies a different strategy.

Gotchas

Concurrent writes: if multiple agents log failures simultaneously, you can get race conditions. Use file locking (fcntl on Unix, msvcrt on Windows) for production use. Here's a minimal lock wrapper:

import fcntl
def safe_write(path, data):
    with open(path, 'w') as f:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
        f.write(data)
        fcntl.flock(f.fileno(), fcntl.LOCK_UN)

Library bloat: failures accumulate over time. Add a TTL (e.g., 30 days) and prune old entries. I run a cleanup cron job every Sunday:

# Prune failures older than 30 days
python -c "
import json, time
from pathlib import Path
lib = Path.home() / '.mcp' / 'failures.json'
if lib.exists():
    failures = json.loads(lib.read_text())
    cutoff = time.time() - 30 * 86400
    fresh = [f for f in failures if f['timestamp'] > cutoff]
    lib.write_text(json.dumps(fresh, indent=2))
    print(f'Pruned {len(failures) - len(fresh)} old entries')
"

False positives: an entry marked "fixed" might not work in a different context. Always include the environment details (OS, library version) in the task description. I use a composite key: "browser_navigate|linux|playwright-1.40".
Network tools: the library only works for local failures. For API errors, you need a centralized store (Redis, SQLite). I sync local failures to a Redis instance every 5 minutes for cross-machine sharing.

What about you?

Have you built something similar — a way for your AI agents to share knowledge about failures? Or do you rely on a different pattern for error recovery? I'm curious whether the JSON-file approach scales to larger agent fleets or if a proper database is needed from day one.

Drop a comment below — I read every response.

My MCP Server Kept Crashing. Here's the Error Recovery Pattern That Saved It.

Chen Yuan — Tue, 14 Jul 2026 06:48:08 +0000

I spent three days wondering why my MCP server would just... stop. No crash logs. No error messages. Clients connected fine, then after a few hours, every tool call returned silence.

Turns out the Model Context Protocol (MCP) spec doesn't force you to handle errors — it assumes you will. But the reference implementations are minimal. Your server starts healthy, then bit by bit, things go wrong. A network blip. A malformed tool argument. An external API timeout. And suddenly your AI agent is staring at a blank response.

Here's the pattern I ended up with. It's not clever. It just works.

The Fix

Start with a wrapper around your tool handlers. Every MCP server framework has some kind of tool registration — this works for the official Python SDK, the TypeScript SDK, and most community frameworks:

from mcp.server import Server
from mcp.types import ErrorData, INTERNAL_ERROR, INVALID_PARAMS
import traceback
import json

class ResilientMCPServer(Server):
    """An MCP server that doesn't silently die."""

    async def call_tool(self, name: str, arguments: dict):
        try:
            result = await super().call_tool(name, arguments)
            return result
        except (ConnectionError, TimeoutError) as e:
            # Network-level issues — reconnect and retry
            self._reconnect()
            return self._error_response(
                f"Connection lost while executing {name}: {e}"
            )
        except ValueError as e:
            # Bad arguments from the client — tell them clearly
            return self._error_response(
                f"Invalid arguments for {name}: {e}",
                code=INVALID_PARAMS
            )
        except Exception as e:
            # Everything else — log, don't crash
            traceback.print_exc()
            return self._error_response(
                f"Tool {name} failed: {e}",
                code=INTERNAL_ERROR
            )

    def _error_response(self, message: str, code: int = INTERNAL_ERROR):
        return {
            "content": [{"type": "text", "text": f"ERROR: {message}"}],
            "isError": True
        }

    def _reconnect(self):
        """Reset transport layer without restarting the server."""
        # Your reconnection logic here
        pass

The key detail most people miss: set isError: True in the response. Without it, the client treats your error message as a successful result. Your AI starts hallucinating based on garbage text.

Let me break down what each exception type maps to in practice.

ConnectionError / TimeoutError — These happen when your MCP server talks to an external API or database. The STDIO transport itself won't throw these, but your tool handlers will when they call out to the internet. I reconnect the transport layer instead of restarting the process, which drops the ongoing request but keeps the server available for the next one.

ValueError — Your AI client sent garbage arguments. Maybe it invented a parameter name, passed a string where a number was expected, or sent an empty array. Instead of crashing, tell it exactly what was wrong. I've seen Claude and GPT both self-correct on the next tool call when they get a clear error.

Exception (catch-all) — Everything else. Third-party SDK threw something unexpected. A file was missing. Memory ran out. The catch-all logs the full traceback so you can fix it later, returns a clean error to the client, and keeps the server running. Your agent loses one response but not the whole session.

Why This Works

MCP has a built-in error reporting mechanism through isError, but the SDKs don't enforce it. If your tool handler throws an unhandled exception:

The Python SDK catches it and sends a generic "Internal server error" with isError: True
But if you catch the exception and return a plain text response without isError, the client has no idea something went wrong
The AI happily parses "ERROR: something broke" as if it's legitimate output

The wrapper pattern guarantees three things:

No silent crashes — every exception gets caught at one layer
Client gets signal — isError: True tells the LLM "this response is an error, don't use it"
Server stays up — instead of crashing on the first hiccup, the server keeps serving

I've been running this pattern on two production MCP servers for three weeks. Zero silent failures since the deployment. Before the wrapper, I was getting about one per day.

Gotchas

Don't catch everything blindly. Some errors should crash the server — config validation failures, missing environment variables, corrupted state. I use a separate FatalError exception class for those:

class FatalError(Exception):
    """Errors that should kill the server process."""
    pass

# In the wrapper: re-raise FatalErrors
except FatalError:
    raise  # Let it crash — better than running corrupted

The rule of thumb: if the error means all future requests will also fail, crash. If it's transient or specific to one request, catch it.

Be careful with reconnection in shared-state servers. If your MCP server maintains in-memory state (caches, active connections), a reconnect might leave stale references. I added a health_check tool that clients can call to verify the server is in a good state:

@server.tool()
async def health_check() -> dict:
    return {
        "status": "ok",
        "connections_active": len(pool._connections),
        "cache_size": len(cache._store)
    }

This is especially useful when using MCP over Streamable HTTP transport. A transport-level reconnect resets the HTTP connection but leaves your in-memory caches intact — which is exactly what you want, as long as those caches aren't holding stale connections.

The isError flag is per-response, not per-stream. Each tool call response can independently signal success or failure. This means partial failures are possible — handle them explicitly rather than aborting the whole batch. If one of your tools queries five APIs and three fail, you can return:

{
    "content": [
        {"type": "text", "text": "Fetched data from API-A, API-B, API-C. API-D and API-E timed out."}
    ],
    "isError": True
}

The client gets the partial data and a clear signal that something failed. The LLM can decide to retry the failed ones or proceed with what it has.

One More Thing: Logging

Don't underestimate how much a structured log helps when things go wrong. I added a simple JSON logger that captures every tool call and its outcome:

import logging
import json

mcp_logger = logging.getLogger("mcp.server")
handler = logging.FileHandler("mcp_errors.log")
handler.setFormatter(logging.Formatter(
    '%(asctime)s | %(levelname)s | %(message)s'
))
mcp_logger.addHandler(handler)

# In your wrapper
mcp_logger.error(json.dumps({
    "tool": name,
    "error": str(e),
    "type": type(e).__name__
}))

This saved me twice already — once when I spotted a pattern of timeout errors pointing to a rate-limited API, and once when a ValueError from the client revealed a prompt that was generating malformed tool calls.

What about you?

Have you run into silent MCP server failures? How do you handle tool-level errors in your agent infrastructure? I'm still figuring out the edges — especially around long-running tools that timeout mid-execution.

I Migrated My MCP Server From STDIO to Streamable HTTP. It Almost Worked.

Chen Yuan — Mon, 13 Jul 2026 13:56:55 +0000

Last week I migrated a production MCP server from stdio to the new Streamable HTTP spec. Six hours, fourteen bugs, and one accidental denial-of-service attack later, it was working.

If you are running an MCP server today, you are probably still on stdio. The client spawns your server as a subprocess and talks over stdin/stdout. It works fine for local tools in Claude Desktop or Cursor. But the moment you want to run your server remotely on a VM, behind a load balancer, or as a shared team service, stdio does not cut it. You need HTTP.

Streamable HTTP is the new transport spec that lets clients connect over HTTP with SSE for streaming responses. The SDK added support for it in late 2025, and it is quickly becoming the standard for production MCP deployments. But migrating from stdio is not as simple as swapping a transport class. Here are the six things that broke when I tried.

Problem 1: Session State Does Not Exist Anymore

On stdio, the process IS the session. When your server starts, the client spawns it fresh, sends a few messages, and kills it. Each connection gets a clean slate with no shared state.

With HTTP, the server stays alive. Multiple clients can connect to the same process. The initialize handshake now means something different.

const server = new Server({
  name: "my-server",
  version: "1.0.0"
})

server.setRequestHandler(InitializeRequest, async (request) => {
  currentClientName = request.params.clientInfo.name
})

I had session-scoped caches keyed by a global variable. Second client connected? First cache got silently nuked. Tools started returning stale or empty results for no obvious reason.

The fix was trivial. Key everything by session ID:

const sessionCaches = new Map<string, Map<string, any>>()

server.setRequestHandler(InitializeRequest, async (request, extra) => {
  const sessionId = extra.sessionId
  if (!sessionCaches.has(sessionId)) {
    sessionCaches.set(sessionId, new Map())
  }
})

function getSessionCache(sessionId: string): Map<string, any> {
  return sessionCaches.get(sessionId) ?? new Map()
}

If you do not know to look for this, it creates a maddening bug where the server randomly forgets configuration between requests.

Problem 2: SSE Backpressure

Streamable HTTP uses Server-Sent Events for tools that take a long time to respond. The client POSTs a request, gets back a 202 with a Location header pointing to an SSE stream, and listens for the result.

const response = await fetch("http://server:3456/mcp", {
  method: "POST",
  body: JSON.stringify(request)
})

const stream = new EventSource("http://server:3456/mcp/stream/abc-123")
stream.addEventListener("message", (event) => {
  const data = JSON.parse(event.data)
})

Clean on paper. But when the client reads slowly, the SSE buffer grows. When it hits the Node.js 16KB default highWaterMark, your HTTP server starts rejecting connections.

I discovered this when my web scraper tool started failing randomly under parallel loads.

const activeStreams = new Map<string, AbortController>()

server.setRequestHandler(ToolCallRequest, async (request, extra) => {
  const streamId = extra.streamId
  const controller = new AbortController()
  activeStreams.set(streamId, controller)

  request.onerror = () => {
    controller.abort()
    activeStreams.delete(streamId)
  }
})

I bumped the buffer to 64KB and added a 60-second idle timeout per stream.

Problem 3: Three Different Timeouts

On stdio, client and server share a process lifetime. No network timeout to worry about.

On HTTP, you need to configure at least three:

Connection timeout - how long to keep the TCP connection alive (default: 60s)
SSE keepalive - how often to send a heartbeat so the client knows the server is alive
Tool execution timeout - how long a single tool call can run before the client gives up

The default SSE keepalive interval is 30 seconds. If your tool runs for 45 seconds, the client disconnects before getting a result. No error, no retry, just a hanging request.

const transport = new StreamableHTTPServerTransport({
  sseOptions: {
    keepaliveInterval: 15_000,
    keepaliveTimeout: 10_000,
  },
  httpOptions: {
    requestTimeout: 300_000,
  }
})

Keepalive needs to be shorter than your shortest long-running tool.

Problem 4: CORS Is Real Now

When your MCP client runs in a browser or Electron app, CORS becomes your problem.

import cors from "cors"

app.use(cors({
  origin: [
    "http://localhost:3000",
    "https://claude.ai",
    "app://obsidian.md",
  ],
  methods: ["GET", "POST", "OPTIONS"],
  allowedHeaders: ["Content-Type", "Accept", "MCP-Version"],
  credentials: true,
}))

Claude Desktop sends from localhost:3000, Cursor from file://, Obsidian Copilot from app://obsidian.md. Your server needs to allow all of them.

Problem 5: The Accidental DDoS

On stdio, one request = one process. Cleanup is automatic.

On HTTP, I had a setInterval polling PostgreSQL every 30 seconds. After fifty parallel connections I had fifty queries running every thirty seconds. The pool saturated and down went the server.

// Bad: one timer per connection
server.setRequestHandler(InitializeRequest, () => {
  setInterval(() => warmCache(), 30_000)
})

// Fix: singleton timer
let cacheWarmInterval: NodeJS.Timeout | null = null
server.setRequestHandler(InitializeRequest, () => {
  if (!cacheWarmInterval) {
    cacheWarmInterval = setInterval(() => warmCache(), 30_000)
  }
})

Audit every timer. If it does not need to be per-session, make it a singleton.

Problem 6: Graceful Shutdown

Kill a stdio process and the OS cleans up. Kill an HTTP server with active SSE connections, and clients hang for minutes.

process.on("SIGTERM", async () => {
  console.log("Shutting down. Draining", activeStreams.size, "connections")

  for (const [id, controller] of activeStreams) {
    controller.abort()
  }
  activeStreams.clear()

  await transport.close()
  await new Promise(r => setTimeout(r, 2000))
  process.exit(0)
})

The MCP spec recommends SIGTERM and draining. Do not skip it.

What I Still Have Not Figured Out

Load balancing. With stdio this was not a question. With HTTP, I need sticky sessions or a shared state store. Neither feels right for a twelve-tool MCP server.
Rate limiting. Without process-per-request isolation, one abusive client can starve the others.
Auth. The spec leaves it to the implementer. Using a bearer token for now, but it feels fragile.

If you have dealt with any of these - especially load balancing - I would love to hear how you handled it. The MCP transport spec is evolving fast, and right now a lot of us are figuring out the stdio-to-HTTP migration at the same time.

I Built an AI Agent With Tools. The First 50 Calls Were a Disaster.

Chen Yuan — Sat, 11 Jul 2026 14:30:54 +0000

The Demo That Almost Lost Me the Project

I spent two weeks building the agent. It could look up customer data, calculate metrics, and suggest actions based on real-time API calls. On demo day, the VP asked one question: "What's our churn rate this quarter?"

The agent called the customer search API instead of the analytics API. It passed "churn rate" as a customer ID. When that failed with a 400, it tried multiplying three random metrics together -- average session length times number of users divided by page load time. Six API calls, twelve seconds, about fifty cents in compute. The answer it returned: "Churn rate is approximately purple."

I closed my laptop and said the internet was down.

This is the moment every AI agent builder faces. The model can reason, write code, and plan -- but give it real tools and everything falls apart. I've spent the last three months fixing exactly these failure modes. Here's what I found running over 10,000 tool calls across multiple production systems.

What 500 Tool Calls Taught Me

I started by instrumenting everything. Every tool call logged: which tool was selected, the arguments passed, whether execution succeeded, and what the model did next. After 500 calls through a naive implementation:

Wrong tool selection (23%): The model called get_customer when the question was about subscription data.
Malformed arguments (15%): Missing required fields, wrong types, hallucinated field names.
Failed execution with no recovery (12%): API returned 429, model retried same endpoint 3x without waiting.
Straight-up refusals (3%): Model decided to explain what it would do instead of calling any tool.
Success (47%): Got the right answer on the first try.

Less than half the calls worked. That's not production-ready.

Pattern 1: Validate Args Before They Touch an API

The simplest fix with the biggest impact: validate tool arguments before they reach any external system.

function validateToolArgs(name, args) {
  const schema = schemas[name]
  const errors = []
  for (const [key, def] of Object.entries(schema.properties)) {
    if (schema.required?.includes(key) && args[key] === undefined) {
      errors.push(`Missing required: ${key}`)
    }
    if (args[key] !== undefined && typeof args[key] !== def.type) {
      errors.push(`${key} must be ${def.type}, got ${typeof args[key]}`)
    }
  }
  return errors.length ? { valid: false, errors } : { valid: true }
}

This caught 80% of argument errors immediately. Success rate went from 47% to 67%.

Pattern 2: Run Independent Tool Calls in Parallel

async function runToolCalls(toolCalls, timeoutMs = 10000) {
  const results = await Promise.allSettled(
    toolCalls.map(tc => {
      const check = validateToolArgs(tc.name, tc.args)
      if (!check.valid) return { error: check.errors.join('; ') }
      return callTool(tc.name, tc.args)
    })
  )
  return results.map((r, i) => ({
    id: toolCalls[i].id,
    status: r.status === 'fulfilled' ? 'ok' : 'failed',
    data: r.status === 'fulfilled' ? r.value : r.reason?.message
  }))
}

Parallel calls plus validation pushed success rate from 67% to 83%.

Pattern 3: Exponential Backoff, Handled by the Host

I gave the model a "retry" tool initially. It was a disaster. The model called retry instantly (no delay), called retry on calls that succeeded, and once got stuck in a loop calling retry on retry.

async function robustCall(name, args, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await callTool(name, args)
    } catch (err) {
      if (attempt === maxRetries)
        return { error: `Failed after ${maxRetries}: ${err.message}` }
      const delay = Math.pow(2, attempt) * 1000 + Math.random() * 500
      await new Promise(r => setTimeout(r, delay))
    }
  }
}

The critical insight: the model should never handle retry logic. Your code does it.

With retry plus backoff, success rate reached 91%.

Pattern 4: Be Verbose With Tool Descriptions

The remaining failures were almost all wrong tool selection. Not "Get customer subscription" but:

"Use this when you need to know what plan a customer is subscribed to, their billing cycle, payment method on file, or subscription status (active/cancelled/past_due). Include a customer_id."

I also started merging conceptually related tools. Instead of three separate functions for "get transactions," "get refunds," and "get invoices," I created getCustomerFinancialData(scope, customerId) with three scopes.

This pushed success rate to about 95%.

Pattern 5: Log Everything

This is the meta-pattern that made the other four possible. Every tool call went into a structured log. After three weeks of this, I had a dataset of failure modes from 10,000+ logged calls. Without the logs, I'd be guessing. With them, I could prioritize the patterns that would have the biggest impact.

The tool you need most in an AI agent isn't a search tool or a calculator. It's observability.

What I Still Don't Know

Tool drift at scale. With 30+ tools, accuracy drops off a cliff.
Cost of parallelism. More parallel calls means more API spend per query.
Cross-model consistency. GPT-4o, Claude, and Gemini all handle tool calling differently.

If you've figured any of these out, I'd genuinely love to hear how. Tool calling is the biggest bottleneck in production AI agents right now.

The Old Way of Publishing MCP Servers Is Gone. Here's What Replaced It.

Chen Yuan — Sat, 11 Jul 2026 14:25:42 +0000

I Assumed It Would Be Easy

I've been building MCP servers for a while now. Filesystem tools, search utilities, API wrappers — the pattern is familiar: write the server, test it locally, document it, call it done. The last step — making it discoverable — always felt secondary.

Back when the MCP Registry was a GitHub repo, "publishing" meant opening a PR and waiting. Sometimes days, sometimes weeks. It worked, but it was slow.

Then I heard the registry had moved. The new system lives at registry.modelcontextprotocol.io — a proper API-based registry with a CLI tool, OAuth authentication, and namespace verification. No more PR queues. No more JSON files in someone else's repo.

I was excited. Then I tried it.

What the Registry Actually Is Now

The new MCP Registry is a standalone service built in Go, backed by PostgreSQL. It doesn't host your server code — just metadata. Your actual package lives on npm (or PyPI, NuGet, etc.), and the registry stores a pointer with configuration details.

To publish, you use a CLI tool called mcp-publisher. The flow goes like this:

Add an mcpName field to your npm package.json
Publish your package to npm
Install mcp-publisher (a single Go binary)
Run mcp-publisher init to generate a server.json template
Run mcp-publisher login github for authentication
Run mcp-publisher publish

On paper, six steps. In practice, I hit unexpected walls at almost every one.

The First Wall: Namespace Verification

The registry uses a reverse-DNS naming convention. Your server name looks like io.github.your-username/server-name. This prevents naming collisions — nobody can squat on "weather" — but it also means you need to prove ownership of that namespace.

For GitHub-based auth, your server name must start with io.github.<your-github-username>. The mcpName field in package.json needs to match exactly. I initially used my-username/weather and got a generic "Registry validation failed for package" error.

The error message doesn't tell you why it failed. I spent twenty minutes cross-referencing the docs before I spotted the namespace format requirement.

The fix was simple:

{
  "name": "@my-username/mcp-weather-server",
  "mcpName": "io.github.my-username/weather"
}

The Second Wall: Version Alignment

The server.json file has a version field that must match your npm package version. I bumped my npm version from 1.0.0 to 1.0.1 but forgot to update server.json. The CLI didn't throw a clear error — instead, it said "validation failed" with no further detail.

This is the kind of thing that's obvious in retrospect but frustrating in the moment. Both files need to be in sync. I now keep them open side by side and update both before running publish.

The CLI Tool Is Actually Good

Once I got past the configuration issues, the CLI experience was smooth. The mcp-publisher init command scans your project and generates a server.json with reasonable defaults:

{
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "name": "io.github.my-username/weather",
  "description": "An MCP server for weather information.",
  "repository": {
    "url": "https://github.com/my-username/mcp-weather-server",
    "source": "github"
  },
  "version": "1.0.0",
  "packages": [
    {
      "registryType": "npm",
      "identifier": "@my-username/mcp-weather-server",
      "version": "1.0.0",
      "transport": {
        "type": "stdio"
      }
    }
  ]
}

Authentication uses GitHub's device authorization flow — you run mcp-publisher login github, get a code from the terminal, enter it on github.com/login/device, and you're authenticated. No manual token management, no API key generation.

The publish command itself takes about two seconds. Compare that to the old PR process which could take weeks. Speed is the killer feature here.

My Workflow for a Painless First Publish

If you're publishing for the first time, here's what I'd do differently knowing what I know now:

Read the quickstart first — It's at github.com/modelcontextprotocol/registry. The docs are well-written. Don't skip them.
Set up mcpName before you publish to npm — Adding it retroactively means re-publishing your npm package, which creates a new version number. Better to add it from the start.
Run mcp-publisher init and carefully check every field — The generated server.json picks up environment variables from your project. I had to remove an old API key placeholder it found in my .env.
Keep versions in sync — npm version and server.json version must match. I use a post-version script to update both automatically.
Verify after publishing — The API endpoint registry.modelcontextprotocol.io/v0.1/servers?search=your-name lets you confirm your server is listed.

What's Still Missing

The registry is in preview, so gaps are expected. Here's what I wish existed:

Usage metrics — I have no idea if anyone searches for or installs my server. A download count would help me decide whether to keep maintaining it.
Version lifecycle commands — Can I unpublish a broken version? The docs don't mention deprecation or yanking.
A visual directory — The API is great for automation, but a web UI for browsing servers would make discovery much easier.
Better error messages — "Registry validation failed for package" could mean ten different things. The CLI needs to tell you which field failed and why.

Should You Publish?

If you've built an MCP server that you actually use, yes — go through the process. The friction is real but manageable: maybe an hour the first time, then five minutes for subsequent publishes. The registry is becoming the canonical way to make MCP servers discoverable, and clients like Claude Code, Cursor, and VS Code extensions are integrating registry search directly.

I still haven't figured out how to handle version updates cleanly, or whether I should unpublish servers I no longer maintain. If you've been through it, I'd genuinely love to hear how you handle these — drop a comment or reach out.

The old PR-based system is gone. The new one isn't perfect. But it's a real improvement — and it'll keep getting better.

Docker 容器化实战：从零到生产部署

Chen Yuan — Sat, 23 May 2026 12:47:29 +0000

Docker 容器化实战：从零到生产部署

容器化是现代 DevOps 的核心技能。本文从 Dockerfile 编写到 Docker Compose 编排，带你掌握容器化全流程。

1. Dockerfile 最佳实践

多阶段构建

# 构建阶段
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

# 运行阶段
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
EXPOSE 3000
CMD ["node", "dist/server.js"]

减小镜像体积的技巧

# 1. 使用 Alpine 基础镜像
FROM python:3.11-alpine

# 2. 合并 RUN 指令减少层数
RUN apk add --no-cache gcc musl-dev && \
    pip install --no-cache-dir -r requirements.txt && \
    apk del gcc musl-dev

# 3. 使用 .dockerignore
# .dockerignore 内容：
# .git
# node_modules
# *.md
# .env

2. Docker Compose 编排

Web 应用 + 数据库

version: '3.8'

services:
  web:
    build: .
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=postgres://user:pass@db:5432/mydb
      - REDIS_URL=redis://cache:6379
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  db:
    image: postgres:15-alpine
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=mydb
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d mydb"]
      interval: 10s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine
    volumes:
      - redisdata:/data
    command: redis-server --appendonly yes --maxmemory 256mb

volumes:
  pgdata:
  redisdata:

3. 安全加固

非 root 用户运行

FROM node:18-alpine
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
WORKDIR /app
COPY --chown=appuser:appgroup . .
USER appuser
CMD ["node", "server.js"]

镜像扫描

# 使用 Trivy 扫描镜像漏洞
trivy image myapp:latest

# 使用 Docker Scout
docker scout cves myapp:latest

4. 日志管理

# docker-compose.yml 中配置日志
services:
  web:
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

5. CI/CD 自动构建

# .github/workflows/docker.yml
name: Docker Build & Push
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USER }}
          password: ${{ secrets.DOCKER_PASS }}
      - uses: docker/build-push-action@v5
        with:
          push: true
          tags: myapp:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

总结

Docker 容器化的核心要点：

多阶段构建：减小镜像体积
非 root 运行：安全加固
健康检查：确保服务可用
日志管理：便于排查问题
CI/CD 集成：自动化部署

掌握这些，你就能把应用从容开发到生产了！