DEV Community: MilkyWay008

uv lock says "No solution found": how to read the dependency deadlock and break it

MilkyWay008 — Tue, 22 Sep 2026 17:07:21 +0000

You add one library, run uv lock, and the resolver prints 40 lines of "Because ... And because ..." and quits with a non-zero exit code. You did not change any pins. The project was fine yesterday.

I have run into this class of failure a few times and it usually ends the same way: the resolver is right, and one package in the graph has a stale upper bound.

What that wall of text is saying

The first line is the one that matters:

× No solution found when resolving dependencies:

Everything after it is a derivation chain. Each line is a requirement read out of some package's metadata, and the chain ends where two of them want different things.

A real example, from a LangGraph issue (langchain-ai/langgraph#8352): langgraph-api 0.11.0 pins opentelemetry-exporter-prometheus<0.59, which forces opentelemetry-sdk into an old range. Meanwhile pydantic-ai 2.x pulls logfire>=4.35.0, which wants opentelemetry-sdk>=1.39.0,<1.43.0. Those two ranges never overlap, so no set of versions satisfies both.

Nobody is being unreasonable here. That 0.59 bound was probably correct when someone wrote it...... it just never got revisited. The person who filed the issue could not fix it either, because the metadata ships from a private source tree, so only the maintainers can widen the pin.

Two commands make the chain readable:

uv lock -v                            # full derivation, as the resolver sees it
uv tree --invert opentelemetry-sdk    # who wants which version

Read the chain bottom-up, find the two packages that disagree, ignore the middle.

The silent downgrade is worse than the error

In that same issue the resolver could quietly settle on a much older langgraph-api (0.7.27 was observed), and langgraph dev fell back to an older build without complaining. The app boots, on code you did not ask for, and the Studio upgrade banner stays unsatisfiable.

That one costs people days, because nothing looks broken. A successful resolution is not proof you got what you wanted. Check the installed version, not the requested one.

Constraints cannot fix a deadlock (this is the part people get wrong)

uv has two settings for this, both in the workspace-root pyproject.toml under [tool.uv]. They look similar and behave nothing alike.

constraint-dependencies is additive. The docs describe constraints as restricting "the versions of dependencies that are selected during resolution", so they get intersected with whatever the packages already declared.

override-dependencies is absolute. The docs say overrides "force selection of a specific version of a package, regardless of the version requested by any other package, and regardless of whether choosing that version would typically constitute an invalid resolution". The resolution docs call overrides the escape hatch for erroneous upper version bounds.

So when the ranges are disjoint, a constraint can never save you. Constraints only narrow the space, and the space is already empty. This matters because the reflex is to reach for a constraint when the real tool is an override.

[tool.uv]
# stops the resolver from silently picking something ancient
constraint-dependencies = ["langgraph-api>=0.10.3"]

# the deadlock breaker: deliberately contradicts a declared upper bound
override-dependencies = ["opentelemetry-exporter-prometheus>=0.63b1"]

One placement gotcha: uv reads these from the workspace root only. Putting them in uv.toml, or in a member package's pyproject, does nothing.

The order I would try things

1st, pin the thing that moved back to the last version that worked. Boring, safe, usually enough to get moving today.

2nd, add a floor constraint on whatever was being silently downgraded, so you find out about it instead of living with it.

3rd, override the stale bound, then actually run the app. uv lock exiting 0 only proves the graph resolves. The package that asked for the old SDK is now untested territory.

4th, if the two libraries genuinely cannot coexist, give them separate environments. Ugly, but honest, and sometimes correct.

Leave a trail

Keep the override in the file with a comment, a link to the upstream issue, and the date you added it. An override without a note turns into permanent mystery debt six months later. In this case the pin was widened on the 0.13.0 dev line (<0.64,>=0.63b1, landing the SDK at 1.42.1) while stable still carried the old bound, so the plan is: wait for the release, then delete the block.

If you use pip instead of uv, the constraint file (pip install --dry-run -c constraints.txt) is additive in the same way, so it will not break a disjoint range either. Worth knowing before spending an hour on it.

I can be wrong about your particular graph, but from what I have seen the fix is rarely a clever resolver flag. It is reading the chain, working out which bound is stale, and forcing that one thing while you wait for upstream...... then writing down why.

One honest gap: I could not independently confirm whether stable langgraph-api still carries <0.59 at the time of writing. The issue is open. Check it before copying the override.

HTTP 400 from your own LLM endpoint? Check for content: null

MilkyWay008 — Mon, 21 Sep 2026 17:28:55 +0000

Someone I was helping had a tidy little setup: a self-hosted OpenAI-compatible endpoint in front of their model, a coding agent pointed at it, months of smooth sailing. Then they bumped the agent from 7.3.42 to 7.3.44 and every request came back HTTP 400. Same endpoint, same key, same model. Only the client version changed.

The error body was short, and it looked roughly like this:

400 Bad Request
{"error": {"message": "Type validation failed",
           "path": "/messages/2/content",
           "keyword": "invalid_union"}}

If you run a gateway in front of a local model (vLLM, LiteLLM, llama.cpp, Open WebUI, LM Studio, or your own FastAPI shim), this one will find you eventually. The error already tells you where to look, and the fix is usually a couple of lines on whichever side of the wire you control.

The field it names is the whole clue

/messages/2/content means the third message in the array, its content field, and a union check that failed. Validators generated from a schema (pydantic, zod, ajv and friends) hand you the JSON path for free. Read the path first, then read the raw request, and stop guessing about the model.

Log the outgoing body at the gateway (raw JSON, not a redacted summary), find that index, and look at the message. In this case it was an assistant message with tool calls and no text:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {"id": "call_1", "type": "function",
     "function": {"name": "read_file", "arguments": "{\"path\": \"a.txt\"}"}}
  ]
}

Nothing looks broken there. That's the trap.

What changed, and why neither side is crazy

On the client side, Vercel's AI SDK changed tool-only assistant messages from content: "" to content: null on purpose. The commit message spells out why (vercel/ai@bfb756d8, April 2026): providers backed by AWS Bedrock reject an empty text block with ValidationException: text content blocks must be non-empty, so the serializer became content: text || null. A lot of coding agents sit on top of that SDK, which is why the change showed up in several places at once.

On the server side, the endpoint's validator insists that an assistant content is a string. Null isn't a string. Union check fails, request dies.

OpenAI's own schema allows null, though. In the OpenAI OpenAPI spec, the assistant message defines content as string | array of parts | null, described as "The contents of the assistant message. Required unless tool_calls or function_call is specified." The only required property on that object is role.

So content: null on a tool-call-only assistant message is legal, and the validator is the piece that's out of step with the protocol. "My schema says no" is a different sentence from "the spec says no."

Fixing it

If you own the endpoint, normalize at the edge. Accept null when tool calls are present, and rewrite it into something your backend tolerates. A small piece of Starlette middleware is enough:

import json
from starlette.middleware.base import BaseHTTPMiddleware

def normalize(body):
    for m in body.get("messages", []):
        if m.get("role") == "assistant" and m.get("tool_calls") and m.get("content") is None:
            m["content"] = ""   # or " " if your backend also dislikes empty strings
    return body

class NullContentFix(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        if request.url.path.endswith("/chat/completions"):
            payload = await request.json()
            request._body = json.dumps(normalize(payload)).encode()
        return await call_next(request)

That comment about empty strings isn't hypothetical. Continue's OpenAI converter does message.content || " " with the literal comment "LM Studio (and other providers) don't accept empty content." Different clients picked different workarounds for the same edge case, which is a decent hint about how long this has been rattling around. If your backend is picky about empty strings too, coerce to a single space.

If you don't own the endpoint, pin the client. The actual difference between working and broken was @ai-sdk/openai-compatible 2.0.41 versus 2.0.48. A sibling tool still sitting on 2.0.41 was reported working against the same endpoint, which is a reminder that "it works in tool X" says nothing about your config. Pin it in the lockfile, or put your own proxy in the middle to rewrite null.

And don't wait on the upstream fix. A normalizing pull request for the client was opened and closed without merging (kilocode#12331; the original report is kilocode#12330), so as of today it's on you to pin or normalize.

The same 400 in other disguises

Once you've seen one message-shape mismatch, you spot them everywhere. The ones I keep running into:

Reasoning blocks that don't survive a round trip. Open WebUI dropped the Anthropic thinking-block signature when saving reasoning_details, then replayed the unsigned block on the next turn. Anthropic rejects it with Invalid signature in thinking block, so turn 2 dies every time, identically across five providers (open-webui#27467). The fix PR is closed without merging, so stripping unsigned reasoning from prior assistant turns in a proxy is the workaround for now.
Streaming reasoning where delta.content isn't a string. n8n 2.36.8 with the Mistral Chat Model: content arrives as an array of thinking blocks, the node's schema wants a string, and you get invalid_union. Routing through a proxy that strips thinking deltas worked (n8n#37352, which n8n closed on Sep 8).
Extra parameters the client started injecting. One client began sending prompt_cache_breakpoint to a custom Responses endpoint and got 400 invalid_parameter on every request, reproducing on a clean VM (kilocode#13285). Strip the unknown field at the proxy, or pin the client back.
Role remapping that breaks the model's chat template. LM Studio maps each developer message to its own system message, so the second one lands mid-prompt and Qwen's Jinja template throws System message must be at the beginning (lmstudio#2298). Consolidate developer messages client-side, or fall back to /v1/chat/completions.

The habit that saves the evening

When a proxy or router sits in the middle, the 400 is often produced by the translation layer, not the model. Test with tools disabled, try the same model through a different route, and read the JSON path in the error before touching anything else.

Then decide who owns the field, fix the side you control, and pin the version of the thing that changed instead of the whole stack.

And if you happen to be the one writing the validator: accept null on assistant messages that carry tool calls. It's in the spec.

I've only hit this on a handful of stacks, so treat it as a starting point rather than gospel, and test it against your own gateway. I keep notes on this class of failure in a small KB repo, Windows-flavoured more often than not: hermes-kb-hack-fix. Hope it saves you an evening.

Your agent's SQLite state DB keeps corrupting: what actually causes it, and how to recover the data

MilkyWay008 — Sun, 20 Sep 2026 17:20:16 +0000

Your agent's SQLite state DB keeps corrupting: what actually causes it, and how to recover the data

I run an agent that keeps its entire brain in one local SQLite file. Three weeks ago it came back with database disk image is malformed, and I did the first thing most of us do: deleted the -shm sidecar, re-ran the repair, moved on. It came back about twenty hours later, same table, same error.

So I finally went and read SQLite's own list of ways you can corrupt a database file. It's a good list. Nearly everything on it is something we do to the database, not something the database does to itself.

Here is the order I wish I had done this in.

Read the error properly first

Two different messages get called "corruption" and they are not the same thing.

database disk image is malformed is SQLITE_CORRUPT (11): page-level damage inside a file that is still recognisably SQLite.

file is not a database is SQLITE_NOTADB (26): the header isn't SQLite at all, so it's the wrong path, a zero-length file, or something else wrote over it.

If you only read one error, read the first one. By the time you reach the last error in a stack trace, the real cause is usually several layers down.

Check the file before you touch it

sqlite3 state.db "PRAGMA integrity_check;"
sqlite3 state.db "PRAGMA quick_check;"
sqlite3 state.db ".dbinfo"

integrity_check returns ok, or a list of problems. It stops after 100 of them, so a long list means "lots", not "exactly 100". quick_check runs the same test minus the table-versus-index comparison. It's fast and it can miss damage. And PRAGMA foreign_key_check is not a corruption test at all, it only reports FK violations.

If the output mentions ptrmap, that's the pointer map that autovacuum keeps, and it now disagrees with what is actually stored on the pages. That is structural damage from a stray, short or duplicated write, which in practice usually means two writers on one file. It is not an index logic bug, and no amount of REINDEX will clear it.

What actually causes it

SQLite keeps a canonical list at https://www.sqlite.org/howtocorrupt.html. Ranked by how often I've seen each one show up in agent and desktop app trackers:

Deleting or renaming -wal / -shm while something still has the database open. The -shm is the wal-index, the shared memory that coordinates readers and writers. Unlink it under a live connection and that coordination is gone. This is the one I did.
Putting the state file on a network or synced folder. NFS, SMB, OneDrive, Dropbox, iCloud. SQLite's locking is advisory and assumes a real local filesystem with working locks. A sync client rewriting pages underneath it is a corruption machine.
Two processes writing the same file without agreeing on locking. A gateway plus a CLI plus a second server pointed at one state directory. The app should own one writer, or funnel everything through a single process. That's an application bug, not a SQLite one.
Copying a live database with a file copy instead of the backup API or VACUUM INTO. Same category: deleting a hot journal, or restoring a backup while a transaction is still open.
PRAGMA synchronous=OFF, or a disk that reports a write as synced when it isn't.

Real SQLite bugs do exist, but they are narrow and old: a WAL race writing to a WAL-mode database (§8.1), and corruption after switching between rollback and WAL mode with a VACUUM in between (§8.4). If your database corrupts on a local disk, with one writer, no sidecar fiddling and a sane busy_timeout, you're in that territory, and it's worth collecting a repro instead of re-reading your own code.

Recovering the data

There is no in-place repair. Anyone telling you to "fix" the file is guessing. The steps below are what worked for me on an agent that had two processes pointed at the same state directory, and I can't promise they map one-to-one onto your setup, but there's nothing destructive in them. Try them before you delete anything.

Stop every writer. Kill the app, the gateway, the CLI. If a process still holds the file, everything below is theatre.
Copy the whole set to scratch: state.db plus state.db-wal plus state.db-shm. Copying the .db on its own throws away committed data that only lives in the WAL, and it can make a healthy database look corrupt.
Dump and reload into a new file:

sqlite3 corrupt.db ".recover" | sqlite3 new.db

.dump stops at the first sign of corruption. .recover keeps going, reassembles what it can from the surviving pages, and parks rows it cannot attribute into a lost_and_found table.

Validate before you swap. Run PRAGMA integrity_check; and PRAGMA foreign_key_check; against new.db, then REINDEX;. Keep the old file around until you're sure.
Count what you lost. Rows on damaged pages are gone. Uncommitted WAL transactions are gone. Check the lost_and_found row count, and diff the schema against what you expect, because a table whose root page went missing can disappear silently rather than error out.

Stopping it from coming back

Set PRAGMA journal_mode=WAL;, PRAGMA synchronous=NORMAL;, PRAGMA busy_timeout=5000; and actually handle SQLITE_BUSY instead of swallowing it. Ignoring busy is on SQLite's own corruption list.

Never delete or rename -wal or -shm while any connection is open. The checkpoint and the cleanup happen when the last connection closes. Let it happen.

Keep one writer per state directory, enforced with a lock file if the app can't guarantee it, or do all the writing in one process.

Keep the state directory out of OneDrive, Dropbox, network shares, and aggressive antivirus realtime scanning. Exclude it explicitly rather than hoping.

Don't churn journal modes, and don't schedule VACUUM against a live, high-churn database. Back up with VACUUM INTO backup.db or the backup API, nightly, before you need it.

The honest part

I'm not going to pretend this explains every case. If a freshly created database corrupts on local disk with one writer, one process and no sidecar deletion, that's a genuine bug and it deserves a repro with the file attached, not a checklist. Watch for one more trap there: VACUUM INTO is transactional for the source, but an unplanned shutdown can still leave the output file incomplete. Validate the backup you just made, don't assume it.

The rest of it is mostly ours. The file was fine, and we kept pulling the rug out from under it. One writer, local disk, hands off the sidecars, and a backup you have actually restored at least once. That's how I run it now, and the malformed errors stopped.

The tool is installed and on PATH. Your app still can't find it.

MilkyWay008 — Sat, 19 Sep 2026 17:10:17 +0000

You install a tool. You open a fresh terminal, run where ffmpeg, and it prints a path right away. Then you go back to your IDE, or your desktop app, or your agent, and it tells you the tool isn't installed. Same machine, same PATH, two different answers.

I've hit this twice in the last month: once with git inside a code agent's worktree manager, and once with ffmpeg inside an MCP subprocess. Both people had already checked PATH, so they figured the app was broken. Most of the time it isn't. Windows handed that app a copy of the environment from an earlier point in time, and nothing ever told it things changed.

Every process gets a frozen copy of the environment

Windows builds an environment block when it creates a process. Microsoft's docs are blunt about it: "By default, a child process inherits the environment variables of its parent process." So your app, the helper processes it spawns, the extension host it loads and every tool it shells out to all keep the block they were born with. Editing PATH afterwards doesn't reach back into a running process.

When you add a directory to PATH, Windows writes it to the registry (machine-wide under HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment, per-user under HKCU\Environment) and broadcasts a WM_SETTINGCHANGE message with lParam set to the string Environment. The docs say that broadcast is what lets "applications, such as the shell, to pick up your updates." The word to notice there is shell. Explorer refreshes its environment, and anything Explorer launches after that inherits the fresh copy. Anything already running keeps the old one, and so does everything it spawns from then on.

That's the whole mechanism. There's no cache to clear and no config to reload.

Diagnosing it in about a minute

Open a new terminal window and run where ffmpeg (or where git). If that fails too, the app isn't your problem. Your PATH edit didn't stick.
Check when the app actually started:

Get-Process -Name Code,cherry-studio -ErrorAction SilentlyContinue | Select-Object Id, ProcessName, StartTime

If that start time is older than your PATH edit, you can stop here.

Ask the app what it saw. Plenty of tools log the lookup they performed: an IDE extension's output channel, or an MCP server's stderr line like FFmpeg not found at: ffmpeg.
Look at both halves of PATH, because Windows combines them when it builds the effective value for a new process:

[Environment]::GetEnvironmentVariable('Path','Machine')
[Environment]::GetEnvironmentVariable('Path','User')

Machine entries come first in the combined list, so if two directories contain a copy of the tool, the earlier one wins. It isn't always the copy you meant.

Restart the process, not the window

Quit the app completely, then check Task Manager for leftovers: extension hosts, tray helpers, node or electron children, sidecars. A lot of desktop apps are single-instance, so reopening while the old process is still alive just shows you the same stale window with the same stale environment. Kill the leftovers first.
If you launched the app from a terminal, close that terminal as well. It took its environment when it opened, and the app inherited it from there. A shell started from another shell has the same problem one level up.
Log off and back on if the change was machine-wide and the app is stubborn. You get a fresh Explorer and a fresh session environment instead of fighting the old one.
Services and scheduled tasks get none of this. Their host processes started at boot, so they keep the PATH from boot until you restart the service or reboot. Logging off doesn't touch them.

When the restart doesn't help

The git thread above is the awkward version: git worked in VS Code's integrated terminal, worked in Visual Studio, worked when typed by hand, and the extension still said "Git is not installed or found in PATH." Restarts and a reboot changed nothing.

When the environment is definitely fresh and the tool still isn't found, the app isn't using PATH the way you think it is. The usual reasons:

It runs its own detection with a hardcoded list of locations instead of consulting PATH, and your install isn't on that list.
It resolves the wrong name. Git for Windows ships git.exe under both cmd\ and bin\, and a tool pointed at one of them while looking for the other will fail even though git works fine for you. On Windows a spawn without a shell doesn't apply PATHEXT either, so a bare git can fail where git.exe succeeds.
There's more than one copy. Run where git and count the lines. A PortableGit bundled inside another client, a WSL git and your own install are all candidates, and the first one in the combined PATH wins.

The durable fix is to stop relying on inheritance. Point the app at the absolute path in its own settings (a git.path style key, or whatever the equivalent is), or give it an explicit environment block. For an MCP server that's normally an env entry in the config:

"env": { "PATH": "C:\\ffmpeg\\bin;<your existing PATH here>" }

Some clients expand ${PATH} for you and some don't, so check yours before assuming either way.

One thing to watch if you edit PATH with setx: it crops values at 1024 characters, silently, so a long PATH loses entries off the end. The documented maximum size of a single environment variable is 32,767 characters, and if you're anywhere near that, something else in your setup needs a look.

Where this usually lands

Restarting an app solves this more often than it sounds like it should, but restart means the process, not the window. If a full restart and a reboot both fail, stop suspecting the environment. The tool is probably fine, and the app's own lookup logic is what to read next.

Both cases came from real threads: the git one is Kilo-Org/kilocode#13452 and the ffmpeg one is CherryHQ/cherry-studio#18528. I've only had eyes on those two, so your mileage may vary. Hope this helps.

Socket error 10013 on Windows: your port is reserved, not blocked

MilkyWay008 — Fri, 18 Sep 2026 17:08:26 +0000

If a dev tool on Windows has ever failed on you with this line:

An attempt was made to access a socket in a way forbidden by its access permissions. (os error 10013)

...your first instinct is probably the firewall, or antivirus, or "I need to run this as admin". I've gone down that road. Turned Defender off, opened an elevated terminal, got the exact same error. Most of the time it's something more boring: Windows has already given that port away, and nothing in the error message says so.

Two errors that look alike and mean opposite things

If you spend any time in Windows socket logs, you'll keep meeting two of them.

10048 (WSAEADDRINUSE) means something is already listening on that port. netstat -ano | findstr :47311 gives you the PID, and killing that process fixes it.
10013 (WSAEACCES) means you aren't allowed to bind there at all. Nothing is listening. The firewall isn't involved, and running as admin usually changes nothing, because the port sits inside a range Windows has reserved.

On any machine running Hyper-V, WSL2 or Docker Desktop, the Host Network Service (WinNAT) books blocks of TCP ports for itself when it starts. Those blocks aren't occupied, they're reserved. A tool that grabs a random port for a local listener, like an OAuth callback, a preview server or a test runner, can land inside one. And since the port gets picked at random each run, the failure comes and goes. That's the part that makes it so annoying to chase.

Step 1: look at the reservations

No admin needed for this one:

netsh int ipv4 show excludedportrange protocol=tcp

You get a table of start ports and the number of ports in each block. On a box with WSL2 or Docker Desktop I usually see a few blocks of 100 somewhere up in the 49000-51000 region, but which bases you get depends on the machine and can shift between boots. Run it fresh instead of trusting output you saved last month.

If you want the ephemeral range as well:

netsh int ipv4 show dynamicport tcp

Then check whether your port falls inside one of the blocks. Everything from startport up to startport + numberofports - 1 is covered.

Step 2: move the port, if you're allowed to

This is the fix that actually stuck for the person in the thread I was reading (github/copilot-cli#4463). Copilot CLI picks its OAuth callback port automatically, and the listener blew up with 10013 before the browser even opened, which is a fun way to spend an afternoon. He listed the excluded ranges, pinned the port to 47000, restarted, and the auth flow completed.

Most tools that do a loopback OAuth flow have a setting for this somewhere. For Copilot CLI it's the auth.redirectPort key in the config:

{
  "auth": { "redirectPort": 47000 }
}

For a plain dev server it's whatever your framework gives you: --port, PORT=, a listen() argument. Pick something clear of every block, re-run show excludedportrange, and confirm nothing covers it.

One thing worth saying out loud. If the tool worked yesterday and fails today, that doesn't mean the problem fixed itself. It picked a different port. It'll be back.

Step 3: if the port has to stay

Two options here, one temporary and one that sticks. Both need an elevated prompt.

The temporary one releases the NAT service's holds around your bind:

net stop winnat
# start your service, or finish the bind, here
net start winnat

Restart-Service winnat does it in a single step if you only need the reservations dropped and re-taken. While winnat is stopped, outbound NAT and port forwarding for WSL2, Hyper-V VMs and Docker Desktop containers degrade, so don't do this on a machine that's actively serving containers. The reservations also come back the next time the service starts, reboot included, and on newer builds the Host Network Service may re-take them almost immediately, in which case this buys you nothing.

The durable option is to make the reservation yours instead:

netsh interface ipv4 add excludedportrange protocol=tcp startport=47311 numberofports=1 store=persistent

Windows now keeps that port out of dynamic allocation, so nothing else claims it and your explicit bind keeps working across restarts. store=persistent is the part that survives a reboot. To undo it later:

netsh interface ipv4 delete excludedportrange protocol=tcp startport=47311 numberofports=1 store=persistent

You are now the owner of a reserved port. That's the whole point, but it's worth remembering you did it, for the day months from now when something else can't have that port.

If the excluded list is empty and you still get 10013

Then it isn't the NAT service. The usual suspects at that point are endpoint security and VPN filter drivers. Some EDR and AV products, plus a few corporate VPN clients, install a filter that refuses binds for specific programs, and the error surfaces as the same 10013. The tell is a binary that fails on one machine and works on another with the same port, along with a block event in the security product's own log. I haven't hit that one often, so I'd read the vendor log before guessing at it.

The short version

I treated 10013 as a permissions problem for longer than I'd like to admit before it occurred to me to just look at the reserved ranges first. It's a two second command, and it tells you straight away whether you're chasing a port collision or something else entirely. Run it before you touch the firewall:

netsh int ipv4 show excludedportrange protocol=tcp

Topic came from a stuck-user thread over on the Copilot CLI repo. The port-pinning fix there is the reporter's, not mine, and it's the one that got him unblocked.

Docker EACCES after an image upgrade: check the container's uid against the mount

MilkyWay008 — Thu, 17 Sep 2026 17:06:59 +0000

A container that worked yesterday doesn't start today. The image tag moved by one patch version, nothing on your side changed, and the logs end with something like:

EACCES: permission denied, mkdir '/root/.flowise'

Or you get the friendlier version. The web UI comes up, the API answers 502, and further up the log there's a Python PermissionError from the entrypoint plus sqlite3.OperationalError: unable to open database file.

Same failure, different outfit. The image changed which user it runs as (or which HOME it uses, or which subfolder of the mounted directory it writes into), and the directory on your host is still owned by somebody else. I've run into this class a few times now...... it's worth knowing the 2-minute diagnosis instead of rolling the tag back and hoping. Rolling back does work, by the way, and it's a fine stopgap. It just comes back on the next upgrade.

Why the uid matters

A bind mount is your host directory, handed to the container as is. No translation layer, no magic.

When the container process runs as uid X and tries to mkdir inside a directory owned by uid Y, the kernel does the normal thing: it checks X against the owner, group and "other" bits of that host inode. Not the owner, not in the group, and no write bit for others, and you get EACCES. That's the entire bug.

Two details that bite people.

Root inside the container is a special case. Linux lets uid 0 bypass a lot of permission checks through capabilities like CAP_DAC_OVERRIDE, so running a container as root often hides this problem instead of fixing it. Then a new image switches to a non-root USER, or you move to rootless Docker or Podman where container root maps to a host subuid, and the bypass that was covering for you is gone.

A mode 700 directory is the ugly case. Only the host owner has any access at all, so every other uid in the container is locked out completely and you get a hard crash instead of a partial one.

Why does an upgrade trigger it? Because the new image changed USER (root to 1000, or 1000 to 10001), changed HOME, or started writing to a path inside the mount that the old image never touched. Nothing on your side moved.

Diagnose it in a couple of minutes

All of these are read-only, safe on a running box.

docker logs <container> --tail 50

Look for the failing path, not just the error. That path is the directory that has to be writable.

docker inspect --format '{{.Config.User}}' flowiseai/flowise:3.1.4

Empty output means the image runs as root. 1000 or 10001:10001 means it doesn't. Run the same command against the old tag and compare. This is usually the moment it clicks.

docker run --rm --entrypoint sh flowiseai/flowise:3.1.4 -c 'id; echo $HOME; ls -ln /root'

That prints the uid/gid, the HOME the image sets, and who owns the target directory. Worth knowing an image can set USER and forget HOME; then tools write somewhere odd and the error points at the wrong place.

Then the host side:

id -u
ls -ln /path/to/host/mount

If the image runs as 1000 and the mount is owned by 1000, permissions are not your problem. Go look at SELinux (more below) or at the app's own config. If the numbers differ, you found it.

The fix ladder

1st, and usually right: make the host directory owned by the uid the container runs as.

sudo chown -R 1000:1000 /path/to/host/mount

Works everywhere, takes seconds. The catch is you've now hardcoded one uid, so the next image that changes its uid breaks you again. Which is the trap you're standing in right now.

2nd, and more durable: use a named volume instead of a bind mount.

docker volume create flowise_data

# docker-compose.yml
services:
  flowise:
    volumes:
      - flowise_data:/root/.flowise

When Docker first creates a named volume and the image has content at that path, it copies the image's own directory into the volume, ownership included. So the volume starts out owned by whatever uid the image expects, and the container can write to it. The costs: your data isn't at a host path you can scp from, and if the volume already exists with a different uid you may have to remove and recreate it (back it up first, please).

3rd: override the user, if you want the container to match your host user.

services:
  flowise:
    user: "1000:1000"

Handy for local dev where you want files to land as you. It backfires when the image's own files belong to a different uid (a home directory owned by node is still not writable by uid 1000), and on Docker Desktop the uid model isn't your host's anyway.

4th: PUID/PGID, when the image implements it. linuxserver-style images read PUID and PGID from the environment and chown their data directories at startup. Only works where it's supported, so check the docs for the image you're actually running.

5th: pin the previous tag. image: flowiseai/flowise:3.1.3. Two minutes of work, unblocks you, fixes nothing, and shows up again on the next upgrade. Fine as a stopgap while an upstream issue gets attention. Not a plan.

Gotchas that will cost you an afternoon

SELinux on Fedora and RHEL. If EACCES survives a correct chown, the directory is missing the container file label. Add :z or :Z to the mount. A correct chown that changes nothing is the SELinux tell.

Docker Desktop on Windows and macOS. Containers there run in a Linux VM with its own uid space, so everything above is for Linux hosts. Desktop layers its own file-sharing quirks on top.

It isn't always the mount root that needs fixing. If the app writes to $HOME/.appname, that subdirectory is the one that has to be writable, and it may be the one that was just created and failed.

chown -R on a big or shared directory is slow, and it re-owns data belonging to other users and apps without asking. On a 200 GB media mount, don't.

One honest caveat

I went and read two of these threads. The one with a proper A/B proof is an agent-canvas image running as uid 10001 against a 700 mount, which booted fine at 777...... that one nails the mechanism. The other, Flowise 3.1.4 failing on mkdir '/root/.flowise', has the exact symptom and a confirmed "revert to 3.1.3 fixes it", but the issue itself has zero comments, so the uid theory there is plausible rather than confirmed. If chown doesn't do it for you on that build, you're probably looking at a different startup bug wearing the same error message.

Either way: next time a container throws EACCES after an upgrade, run the inspect commands before you touch anything. Comparing {{.Config.User}} between two tags takes ten seconds, and it tells you whether you have a permissions problem at all.

Why your MCP server randomly shows no tools: the npx startup tax

MilkyWay008 — Wed, 16 Sep 2026 17:16:18 +0000

If you've wired an MCP server into an agent client with "command": "npx", you've probably met this one. The server runs fine when you launch it by hand. The agent insists the tools don't exist. No error, nothing useful in the log, and maybe it works again tomorrow.

I've been digging through a pile of these reports lately, and I'm fairly sure about the cause: it's usually not your server...... it's the npx layer you launched it with.

What the failure looks like

Three flavors of the same thing, from the threads I read:

The agent announces that the server's tools aren't available, as if you never configured them.
One session works, the next is dead, with nothing changed.
The same server connects fine in the MCP Inspector and fails in the desktop app.

If any of that sounds familiar, the server is probably fine. The spawn is not.

Why npx makes it worse

MCP's stdio transport works by the client launching your server as a subprocess and talking JSON-RPC over stdin/stdout. The spec is blunt about it: "the client launches the MCP server as a subprocess." So startup time is the client's problem, and clients wait with a deadline.

npx -y <package> is not a shortcut to a binary. It boots npm (itself a Node program), resolves the package, consults the npx cache, and only then spawns your server as yet another child process. On a cold cache there's a download in there too. Every step of that happens inside the client's handshake window.

Then the deadline expires. In Copilot CLI's case the handshake budget is reported as a fixed 60,000 ms with no retry. The reporter measured 22 of 76 handshakes blowing past it in a single day (29%). Once a server misses the budget it's marked failed for the entire session, and the agent just says the tools don't exist. That's the silent-no-tools symptom.

The numbers

From the reporter's measurements, same machine, three ways to launch the same server:

How it was launched	Average
`npx -y @azure-devops/mcp`	15.39s (runs: 27.91 / 8.31 / 9.96)
global `.cmd` shim	3.32s
direct `node .../dist/index.js`	3.21s

That's one person, one machine, one day, so read it as a range rather than a law of nature. But the direction matches what I see locally.

My own check, Windows 11, Node 22.23.2, with the package already warm in the npx cache and --offline so nothing was downloaded:

npx --offline -y @playwright/mcp@0.0.81 --version            1.54s / 1.59s / 1.64s
node <npx-cache>/node_modules/@playwright/mcp/cli.js --version   0.44s / 0.44s / 0.48s

With --help, which loads the whole CLI, it was 7.56s versus 1.74s. Booting npm on its own costs about 0.6s on this box; booting node about 0.1s. So even with a warm cache you're paying seconds for nothing...... and a cold cache is a different animal.

Check your own setup first

Time the command your client actually runs. Compare time npx -y <your-server> --help against time node <path-to-entrypoint> --help.
Run the server outside the client. npx @modelcontextprotocol/inspector, then paste the same command in. Works in the Inspector, fails in the app? You have a spawn problem, not a server problem.
Read the client's MCP logs. Claude Desktop writes to %APPDATA%\Claude\logs\mcp*.log on Windows and ~/Library/Logs/Claude/ on macOS. Other clients mostly just say the server failed and stop there.
Ask whether the client retries at all. Most don't. One miss means dead tools for that session.

The fix, in the order I'd trust it

1st, install the server globally and point command at the resolved binary instead of npx. This is the fix the reporter landed on:

npm install -g @playwright/mcp
npm prefix -g                      # where global packages land
where.exe playwright-mcp           # Windows: full path to the shim
command -v playwright-mcp          # macOS / Linux

{
  "mcpServers": {
    "playwright": {
      "command": "C:\\Users\\you\\AppData\\Roaming\\npm\\playwright-mcp.cmd"
    }
  }
}

2nd, skip the shim and hand the entrypoint to node. This was the fastest of the three measured paths:

{
  "command": "node",
  "args": ["C:/Users/you/AppData/Roaming/npm/node_modules/@playwright/mcp/cli.js"]
}

3rd, if you have to keep npx, at least warm the cache. Run the exact command once by hand (npx -y <server> --help) so the package is fetched, then restart the client. It helps, but you still pay the npm boot on every session.

4th, raise the timeout if your client has one. The Copilot CLI issue body says the budget is hard-coded, while a commenter on that same issue says there's a per-entry timeout in the config. I can't settle that from here, so check your own client's docs and version before trusting either claim.

5th, use a remote transport when the server supports it. An HTTP/SSE server is already running, so there's no per-session spawn to be slow. It's not a coincidence that one vendor's resolution was rewriting the desktop app's MCP client instead of fixing anyone's config.

Windows gotchas that masquerade as the same bug

A raw absolute path in command can kill every cold start in Claude Desktop: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'. Use node plus a file path, or an .exe/.cmd, and remember the config is read only at startup, so quit from the tray rather than closing the window.

Also, "command": "C:\\Program Files\\nodejs\\npx.cmd" gets split at the space by the cmd.exe handoff and tries to run C:\Program. The 8.3 short path (C:\\PROGRA~1\\nodejs\\npx.cmd) sidesteps it.

One more config-shape trap, from a different toolkit: if the client defaults to SSE, a stdio-shaped config (command/args) throws URL is required for SSE transport. Declare the transport explicitly and the error goes away.

When it isn't npx

If the server connects and then dies, or the process exits immediately, spawn timing is irrelevant...... go read the server's stderr. A subprocess that hangs past the budget gets treated exactly like a slow one, so a wedged server and a slow launcher look identical from the outside.

None of this is exotic. It's a handshake deadline most clients keep private plus a launcher that's slower than most of us assume. Ten minutes moving your config off npx and you can stop thinking about it.

I built Smart-MCP-Proxy for the adjacent itch, running a lot of servers without restarting the client every time: https://github.com/MilkyWay008/Smart-MCP-Proxy. Worth a look if you've got a wall of MCP entries. Hope this helps somebody.

Your Windows agent CLI pauses 15 seconds before every command: read the Procmon trace

MilkyWay008 — Tue, 15 Sep 2026 17:12:07 +0000

I keep a list of complaints that sound like vibes and turn out to be real bugs. One of them was "the agent CLI on Windows sits there thinking for 15 seconds before it starts". Not slow streaming...... just dead air between Enter and the first byte of actual work, every single command.

Turns out that one has a trace behind it. There's an open issue on the Codex repo (openai/codex#41351) with the kind of evidence I wish every bug report had. Numbers first: about 15.6 seconds per command in the unelevated Windows sandbox, and roughly 122ms for the same command with the sandbox set to danger-full-access.

Same machine. Same binary. A 128x difference.

That is not your CPU, and it's not your disk.

What the trace actually showed

Procmon, filtered to the process and to CreateFile, sorted by duration, puts almost the whole delay in one row: about 15.42 seconds on a single CreateFile, result OBJECT PATH INVALID, on a path that starts with two backslashes and names the NUL device. The stack runs CreateFileW -> GetDriveTypeW -> ZwCreateFile -> FLTMGR.SYS.

Two things stand out. 1st, the path is a device path that isn't spelled the way it needs to be. 2nd, GetDriveTypeW is sitting in the middle of it, and that function is a known hazard on paths it can't classify. Both of those matter, so here's the background.

The path rules that get missed

Microsoft's naming files, paths and namespaces page is worth the 10 minutes if you ever build paths by hand in Windows code. The short version:

NUL is a reserved device name. It works in any directory, which is why > NUL works from anywhere.
\\.\NUL uses the Win32 device namespace. That's the explicit way to open a device.
\\NUL is not a device path. A path that starts with two backslashes is a UNC name, that is \\server\share. So Windows routes it to the network provider chain (LanmanWorkstation, mrxsmb) and goes looking for a server called NUL. There isn't one.
\\?\C:\... is the extended-length prefix. It tells the API layer to skip string parsing and normalization, and it disables device-name translation too.

Then the hazard on top: GetDriveTypeW asks the provider layer what kind of drive a path refers to, and on an unavailable UNC path that call can block for a long time. This isn't theoretical. wxWidgets issue #8859 is the same shape from 2007: wxFSVolume hung for about a minute on an unreachable \\server\drive because its internal FilteredAdd called GetDriveType, and the reporter noted that drive-letter availability gets cached but UNC paths don't, so it hangs on every call. That one got patched in 2022.

So a bogus double-backslash path, plus a function that queries the network provider, equals a stall. The path is the bug. The function is where the time goes.

The actual defect is one over-escaped string

In codex-rs/windows-sandbox-rs/src/acl.rs, the sandbox's allow_null_device() passes this to CreateFileW:

to_wide(r"\\\\.\\NUL").as_ptr(),

That looks like ordinary Windows escaping. It isn't, because r"..." in Rust is a raw string and raw strings don't process escapes. The bytes are literal: four leading backslashes, a dot, then two more backslashes, then NUL. The correct literal for the device namespace has two leading backslashes and one after the dot:

to_wide(r"\\.\NUL").as_ptr(),

I'm not picking on Rust here, the trap exists everywhere verbatim strings do. C# @"...", PowerShell single quotes, raw string literals in most modern languages. You add a layer of escaping to be safe, and the path you ship means something different from the path you meant.

The honest caveat: 15 seconds is not universal

I nearly wrote a wrong paragraph about this, so here it is properly.

On my own Windows 11 box, a malformed device path does not stall. It fails immediately. \\NUL throws out of GetFullPath, CreateFileW returns error 161 (ERROR_BAD_PATHNAME) in under a millisecond, and GetDriveTypeW just says DRIVE_NO_ROOT_DIR. Microseconds, not seconds.

The reporter's machine behaves differently, and their own WPA trace explains why: a third-party filesystem minifilter, 360FsFlt.sys, sits in the path of that failing call and adds about 15.4s of its own. So the malformed path is a real defect either way, but the price of it depends on what is filtering I/O on your machine. Which is also why this is hard to search for. Fast on a clean VM, miserable on a real desk.

What they did confirm with a controlled change is the direction of the fix. Byte-patching that one literal in codex.exe and codex-command-runner.exe, with nothing else changed, took spawn_ready from 23.4s down to 0.6s, then 0.5s and 0.47s on repeats. Same versions, same machine, only the string fixed.

Status while I'm writing this: the issue is still open, no merged upstream fix, and the same literal is still there on main and in the 0.155 alpha I checked. So treat the correction as confirmed by inspection, not shipped.

Diagnosing it on your own machine

This recipe transfers to any "my tool pauses N seconds before doing anything" complaint, which is the main reason I wanted to write it up:

Run Procmon as admin, stop capture (Ctrl+E), clear (Ctrl+X), then start capture just before you reproduce.
Ctrl+L to filter: Process Name is your tool, and Operation is CreateFile.
Options -> Select Columns -> add Duration and Result.
Sort by Duration, descending. One row usually owns the whole delay.
Right-click that row -> Properties -> Stack. If FLTMGR.SYS shows up, you're not looking at app logic anymore, you're looking at a filesystem filter driver.
To name the filter, capture with WPR and read Mini-Filter Delays in WPA, matching the PID and TID from Procmon.
If the stack went through the network provider, correlate the registry and network activity: filter Process Name is System and Path contains LanmanWorkstation.

If your tool stalls and the stack is pure application code, you have a different bug and this writeup won't help you.

A probe you can run in 30 seconds

This demonstrates the path semantics, not the 15 seconds. Add the warm-up call, cold-start noise is real:

Add-Type -Namespace W -Name K -MemberDefinition @'
[DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)]
public static extern uint GetDriveTypeW(string p);
[DllImport("kernel32.dll", CharSet=CharSet.Unicode, SetLastError=true)]
public static extern IntPtr CreateFileW(string p, uint a, uint s, IntPtr sa, uint d, uint f, IntPtr t);
'@

foreach ($p in 'NUL', '\\.\NUL', '\\.\\NUL', '\\NUL') {
  [void][W.K]::CreateFileW($p, 0x60000, 3, [IntPtr]::Zero, 3, 0, [IntPtr]::Zero)  # warm up
  $sw = [Diagnostics.Stopwatch]::StartNew()
  $h  = [W.K]::CreateFileW($p, 0x60000, 3, [IntPtr]::Zero, 3, 0, [IntPtr]::Zero)
  $e  = [Runtime.InteropServices.Marshal]::GetLastWin32Error()
  $sw.Stop()
  "{0,-10} {1,7:N2} ms  GetDriveType={2}  err={3}" -f $p, $sw.Elapsed.TotalMilliseconds, [W.K]::GetDriveTypeW($p), $e
}

On my box that prints err 161 for the over-escaped form and for \\NUL, and err 5 for the two device-namespace forms. Err 5 is not a path failure, it means the path resolved and my probe just didn't ask for the right permissions.

What I'd take away from it

If you build Windows paths by hand, count the backslashes. \\.\ for devices, \\?\ only for extended-length paths, and never a leading \\ unless you actually mean a server.
Don't hand constructed paths to GetDriveTypeW. It asks the provider layer, and that can block.
Stuck on Codex specifically? sandbox = "danger-full-access" under [windows] is the measured escape hatch, about 122ms. It weakens the sandbox, so that's a workaround, not a fix.
Chasing a stall you can't explain? Trace before you theorize. One Procmon row sorted by duration beats an hour of guessing.

I could be wrong about the exact cost on your machine, since all I have is the reporter's trace and my own clean box to compare against. But the path spelling is checkable, and it is wrong upstream, so that part I'm fairly confident about. The trace and the before/after numbers came from the people in that thread, not from me. I just read it and went, huh, that's the trap I've tripped over before.

Docs I leaned on: Naming Files, Paths, and Namespaces, GetDriveTypeW, the Codex issue, and wxWidgets #8859.

JavaScript heap out of memory: read the GC line before you raise the heap size

MilkyWay008 — Mon, 14 Sep 2026 17:20:34 +0000

You know the moment. You reopen a long-running session in your AI coding CLI and instead of your conversation you get a wall of garbage-collector spam that ends like this:

FATAL ERROR: Ineffective mark-compacts near heap limit
Allocation failed - JavaScript heap out of memory
Aborted (core dumped)

Exit code 134, nothing partial to salvage, session unusable. The standard internet answer is "raise --max-old-space-size." I've watched that advice fail more than once, so here's what the message is really saying, and how to tell which of a few very different problems you're actually looking at.

What the crash means

Node aborts when V8's old generation can't grow any further. There are two versions of the message and they mean slightly different things:

Ineffective mark-compacts near heap limit means full mark-compacts are running and reclaiming almost nothing, so V8 stops trying.
Reached heap limit means an allocation failed and the old generation cannot expand.

Both are terminal. Neither tells you why the heap filled up, which is the only part worth knowing.

The ceiling is not 4 GB

If you've read that "Node caps the V8 heap at around 4 GB," that's only half true. V8 derives the default from your physical memory: roughly half of RAM, with a 2 GB floor value as the base default, and the cap only doubles to 4 GB on 64-bit builds with about 16 GB of RAM or more.

Print yours:

node -e "console.log(require('v8').getHeapStatistics().heap_size_limit/1024/1024 + ' MB')"

On the 13 GB Windows box I'm typing this on, that prints 2096 MB. So on a normal laptop the wall is 2 GB, not 4. (The number includes the young generation, so it comes out slightly above whatever --max-old-space-size value is in effect.)

The GC line is the actual diagnosis

When Node dies this way it prints the last few compactions first. Here's a real one from a Copilot CLI crash report:

Mark-Compact 4062.4 (4098.2) -> 4061.5 (4097.5) MB

Read the arrow. 4062 MB in, 4061 MB out. About a megabyte reclaimed out of four gigabytes. That is not a ceiling problem. Those objects are still referenced by the running process, and raising the ceiling just moves the wall a few gigabytes to the right. In that same thread someone tried NODE_OPTIONS=--max-old-space-size=8192 and got the other message instead: Reached heap limit. Same crash, later.

One more thing worth knowing: not every Node CLI honors that variable. Single-executable builds have been reported ignoring NODE_OPTIONS outright, so you can apply the popular fix and change nothing at all.

Three problems wearing the same error message

Retained objects. Reclaim per compact is near zero and heapUsed sits right at the limit. The tool is holding onto history, transcripts, or cached file contents.

A genuinely huge working set. The compacts do reclaim real memory and you still reach the ceiling. Here, raising the limit is the honest answer.

Something that isn't the JavaScript heap at all. rss climbs while heapUsed stays flat. That's native or FFI memory, and the heap flag is irrelevant to it. There's a Kilo CLI report with exactly this shape: RSS at 3.19 GB, peak around 4 GB, then a segfault inside an FFI trampoline after roughly 56 minutes. On Windows you might see the machine go down instead of the process: one Claude Code report has node.exe climbing to 45 GB and hard-locking the box, and raising the pagefile from 2 GB to 16 GB only stretched the crash from 7-13 minutes to about 3 hours. That's memory pressure, not a V8 cap.

Desktop apps add their own multiplier. Base64-inlining an attachment copies the file several times over. One report had a 238 MB PDF turn into about 317 MB of base64, then get copied again by the buffer and the JSON serialization, blowing the heap on an 8 GB machine.

What to try, in order

Read the compact line and your heapUsed before changing any flag. Near-zero reclaim means stop resuming that session.
If it's retained state: start a fresh session and read the old transcript instead of restoring it. Prune or rotate the stored session-history JSON so the next resume loads less. Keep large files out of the process by passing a path rather than inlining the content.
If the working set is genuinely large, raise the cap with headroom: NODE_OPTIONS=--max-old-space-size=6144. The host needs the RAM to back that up.
If rss is the number climbing, stop tuning V8 and go looking for a native leak or a Windows commit-charge problem.
If you're filing an issue, bring evidence with it: run with --heapsnapshot-near-heap-limit=1, open the snapshot in Chrome DevTools, and paste the exact FATAL line, the GC lines, the version, the OS, and whether the session was resumed. That's usually the difference between a report that gets fixed and one that gets closed.

One honest caveat about version bumps. In the Copilot CLI thread the maintainer pointed at a prerelease, a second user reproduced the crash on that prerelease anyway, and the changelog for a later build says long-running sessions "return freed memory to the system instead of holding gigabytes of it." So yes, upstream fixes for this class do exist and ship eventually. But the pointer alone wasn't enough, and there were about a dozen other issues in that repo carrying the same error string.

None of this needs a profiler to start. Read the arrow in the GC line first, because most of these crashes turn out not to be a cap problem at all.

The MCP Python SDK 2.0 broke the wrapper libraries. Here's how to spot it and pin back

MilkyWay008 — Sun, 13 Sep 2026 17:22:32 +0000

If your MCP tools started failing a few weeks ago and nothing in your own code changed, you're not imagining it.

The official Python mcp package shipped 2.0.0 on 2026-07-28. It's a real major version, with public APIs removed and renamed. Latest on PyPI right now is 2.2.0, and a plain pip install mcp resolves to 2.x. Any library that depends on mcp without an upper bound quietly floated forward and broke at install time.

I spent a while reproducing this in throwaway venvs with 1.29.1 and 2.2.0 side by side. Here's what actually changed, what the errors look like, and how to get moving again.

1st, one name collision to get out of the way

Two SDKs share the name. Python mcp is the one on PyPI. The Rust one is rmcp on crates.io, which is what Goose and some other Rust agents use, and it's on 3.x now. If you're debugging a Rust agent, the Python 2.x story is not your story. I nearly chased the wrong version myself...... glad I checked the crate name first.

1st break: streamablehttp_client is gone

In 1.x:

from mcp.client.streamable_http import streamablehttp_client

async with streamablehttp_client(url, timeout=30, sse_read_timeout=300) as (read, write, get_session_id):
    ...

In 2.x that import raises ImportError. The replacement is:

from mcp.client.streamable_http import streamable_http_client

async with streamable_http_client(url) as (read, write):
    ...

Two changes there. The name has underscores now, and the context manager yields a 2-tuple. The 3rd element, get_session_id, is gone.

That 2nd part is the one that hurts, because it doesn't fail at import. The wrapper imports fine, then blows up at runtime, and you get the error a lot of people are probably googling right now:

ValueError: not enough values to unpack (expected 3, got 2)

I reproduced that against a live streamable-HTTP server on 2.2.0. In the installed package it's mcp/client/streamable_http.py, around line 753: yield read_stream, write_stream.

2nd break: the transport kwargs are gone

StreamableHTTPTransport.__init__ in 2.x is just (self, url). The timeout, sse_read_timeout, headers and auth keywords aren't accepted there anymore; set them on the httpx2 AsyncClient instead. Pass them and you get TypeError: unexpected keyword argument 'timeout'.

Honesty note in the other direction: sse_client still takes timeout and sse_read_timeout. So the "all the transport kwargs got dropped" version of this story is wrong. It's the streamable-HTTP path that changed.

3rd break: field names went snake_case

This is the quiet one. The Pydantic protocol models renamed their attributes:

tool.inputSchema goes to tool.input_schema
result.isError goes to result.is_error
structuredContent goes to structured_content
nextCursor goes to next_cursor

The JSON on the wire is still camelCase (the aliases keep that), so servers and clients still agree on the protocol. It's your Python that breaks, with an AttributeError. Bonus trap: model_dump() without by_alias=True now hands you snake_case dicts without complaint, which can poison anything downstream that expects the protocol shape.

Who is actually broken right now

Two examples I verified by downloading the packages rather than trusting a changelog. autogen-ext 0.7.5 declares mcp>=1.11.0 with no upper bound, imports streamablehttp_client, unpacks the 3-tuple, and reads inputSchema / isError, so its [mcp] extra is broken against current mcp, and as of mid-September that's still the case on main. llama-index-tools-mcp 0.5.0 is the same failure with a twist: it declared mcp>=2.0.0 but still unpacked the 3-tuple, so it was broken against its own dependency floor. Fixed in 0.5.1 in late August; 0.6.0 is current.

The pin that fixes it

If you don't need 2.x features, pin back. The official 2.0.0 release notes tell library authors to keep a <2 upper bound, and the 1.x line is still getting security fixes:

pip install "mcp>=1.28,<2"     # resolves to 1.30.0 today
uv add "mcp<2"

If the wrapper already fixed it (llama-index-tools-mcp >= 0.5.1, for example), upgrade the wrapper instead of downgrading mcp. Read the release notes before you assume which side is fixed.

Before guessing, check what you're actually running:

python -c "import importlib.metadata as m, inspect; print(m.version('mcp')); import mcp.client.streamable_http as sh; print(hasattr(sh,'streamablehttp_client')); print(inspect.signature(sh.streamable_http_client))"

And check the wrapper's pyproject.toml for an unbounded mcp dependency. That missing upper bound is the whole bug, and it'll happen again at the next major.

Two things that are not this bug

I saw a couple of claims circulating while digging that can cost you hours if you chase them.

An empty list from a tool producing zero content blocks is not a 2.x regression. I measured the same numbers on both 1.29.1 and 2.2.0: [] gives 0 blocks, '' gives 1, json.dumps([]) gives 1, ['a','b'] gives 2. It's long-standing FastMCP behavior in _convert_to_content, which flattens a list item by item, and an empty list flattens to nothing. structured_content is still correct. If your tool returns a list, return something wrapper-friendly instead of a bare [] so the model gets one text block to read. That one matters if an agent keeps re-running your search instead of concluding "no match".

Separately, 2.x removed automatic return-value wrapping in the low-level Server, and mcp.types moved out into a mcp-types package. Both are in the migration guide.

The takeaway

A dependency major plus an unbounded requirement in the wrapper equals silent breakage at install time, with a stack trace pointing at your code instead of the dependency. Pin it, then skim the migration guide at py.sdk.modelcontextprotocol.io/migration. It's a long read, but the naming section alone saves the afternoon.

I run a pile of MCP servers locally for agent work, and pinning is the boring habit that keeps it quiet. If you want to see how I aggregate and hot-swap those servers, that's in my Smart-MCP-Proxy repo, but the pin is the only part you need today.

And I could be wrong about your specific stack, so test the pin in a throwaway venv first. That's what I did before writing any of this down.

Why your Python tool corrupts text on Windows, and why chcp 65001 doesn't fix it

MilkyWay008 — Sat, 12 Sep 2026 17:08:37 +0000

I lost an afternoon to a config file last week. The graph name was right there in the JSON, spelled exactly the way the CLI wanted it, and the CLI kept saying the graph didn't exist.

Turns out the file wasn't being read wrong...... it was being decoded wrong. The name that came out the other end had a couple of extra characters in it, and nothing raised an error to tell me.

Then the same class of bug turned up in an updater. It printed "Up to date" while two reader threads died with charmap codec can't decode byte 0x81. The check thought everything was fine, which is the worst part of it.

Both are one thing: Python's default text encoding on Windows is not UTF-8. Not for files, and not for subprocess pipes.

What Python actually uses by default

open("config.json") with no encoding= argument decodes with whatever locale.getpreferredencoding(False) returns. On Windows that's the ANSI code page. cp1252 on most en-US machines, cp936/GBK on zh-CN, cp932 on ja-JP, cp1251 on ru-RU.

Pipes are the same story. subprocess.run(cmd, text=True) with no encoding= hands the pipe to TextIOWrapper, which falls back to that same locale codec. So the parent process decodes the child's UTF-8 output with cp1252 and hopes for the best.

If your files are UTF-8 (most config, most code, most modern tooling), you have a codec mismatch. What happens next depends on the bytes, and there are four flavors.

The four ways it breaks

1. Mojibake, no exception

I encoded café— to UTF-8, which gives 63 61 66 c3 a9 e2 80 94, then decoded those bytes as cp1252 and got cafÃ©â€”. cp1252 maps almost every byte to something, so nothing raises. Your dict key is now cafÃ© and the lookup for café misses. That was my graph-not-found bug, and silent corruption is worse than a crash because you get no pointer to it.

2. A hard UnicodeDecodeError

Same bytes, decoded as GBK: 'gbk' codec can't decode byte 0x94 in position 7: incomplete multibyte sequence. GBK wants a valid lead byte followed by a valid trail byte, and a UTF-8 sequence is neither.

3. The single-byte crash

cp1252 defines no character for 0x81, 0x8D, 0x8F, 0x90, 0x9D. One of those lands in the stream and you get 'charmap' codec can't decode byte 0x81 in position 0: character maps to <undefined>. Small detail that explains why this shows up in normal files: 0x81 is a UTF-8 continuation byte, so it appears in the middle of ordinary characters (U+2041 is E2 81 81). You don't need exotic text to hit it.

4. A text file gets called "binary"

Plenty of tools sniff the first N bytes to guess whether a file is binary, and 1000 is a popular sample size. Cut a UTF-8 file at byte 1000 and you can split a multibyte character in half: 'utf-8' codec can't decode byte 0xe6 in position 999: unexpected end of data. The rest of the file decodes cleanly and there is no NUL byte anywhere, but the tool decides it's binary and refuses to display it. I've had that happen on a plain CJK text file.

chcp 65001 won't save you

I tried it. Console code page was 437, I ran chcp 65001, and locale.getencoding() still returned cp1252. getpreferredencoding(False) still returned cp1252. The Popen(text=True) reader still handed me cafÃ©.

chcp sets the console code page. Python's default text encoding comes from the system ANSI code page (GetACP), which is a different thing. And for pipes the console isn't involved at all...... there's no console anywhere in that path.

Fix it in the source

Pin the encoding at every point where bytes become text. That's the whole fix, and it's boring on purpose:

import json, subprocess, sys
from pathlib import Path

# text files: say what they are
with open("config.json", encoding="utf-8") as f:
    data = json.load(f)

text = Path("README.md").read_text(encoding="utf-8")
Path("out.md").write_text(text, encoding="utf-8")

# child processes: text mode, an explicit encoding, and a forgiving handler
p = subprocess.run(["git", "log", "--oneline"], capture_output=True,
                   text=True, encoding="utf-8", errors="replace")
print(p.stdout)

# if it's your own stdout that's misbehaving
sys.stdout.reconfigure(encoding="utf-8", errors="replace")

A couple of details that bite later:

Use encoding="locale" when the file really is in the platform encoding (3.10+). Don't guess.
errors="replace" never raises, which is also why it can hide a real mismatch. Reach for errors="surrogateescape" if you need to round-trip arbitrary bytes without losing any.

To find the sites in code you already have, run with PYTHONWARNDEFAULTENCODING=1 (or python -X warn_default_encoding). Python 3.10+ emits an EncodingWarning everywhere the default is being relied on. Run your tests once with it enabled, fix the hits, done. That's the PEP 597 answer to "how do I find these", and it beats grepping for open(.

If you can't patch the tool

Third-party CLI, vendor binary, something you don't own: set PYTHONUTF8=1 in that process's environment. That's UTF-8 mode (PEP 540, available since 3.7). It makes open() default to UTF-8, switches the filesystem encoding to UTF-8, and, the part that matters here, makes locale.getpreferredencoding() return utf-8 so pipe decoding gets the right codec.

Same script, both settings, on Windows 11 with Python 3.11:

PYTHONUTF8=0  getpreferredencoding: cp1252  |  Popen(text=True): 'cafÃ©â€”'
PYTHONUTF8=1  getpreferredencoding: utf-8   |  Popen(text=True): 'café—'

Notice locale.getencoding() stays cp1252 in both rows. UTF-8 mode doesn't change the machine's ANSI code page, it changes what Python asks for.

Two honest caveats, because this is where people get hurt:

UTF-8 mode is not a codec fix. If the child process actually emits GBK bytes, you still fail. You just fail with utf-8 named in the traceback instead of cp1252. errors="replace" is the crash-proof option. UTF-8 mode covers the common case, it doesn't make you right.
I wouldn't set it machine-wide. It changes the default for every Python process on the box and flips os.fsencode/os.fsdecode to UTF-8, so anything that legitimately exchanges ANSI or OEM encoded data with a Windows component can start producing mojibake. Per process, per venv, or per CI job is the sane scope. Also, PYTHONIOENCODING does not cover pipes: I had it set to utf-8 and the pipe still decoded as cp1252.

For what it's worth, this is going away. PEP 686 makes UTF-8 the default in 3.15, and the same PEP warns that the change can surface UnicodeError and mojibake in code that was leaning on the old behavior without saying so. Expect a wave of these reports when it lands.

The binary-sniff false positive

Don't decide "binary" from a truncated sample. Check for a NUL byte (that's what git does in its 8000-byte scan) and let an incremental decoder deal with the edge, so you never split a character:

import codecs

def looks_binary(path, sniff=8000):
    with open(path, "rb") as f:
        head = f.read(sniff)
    if b"\x00" in head:
        return True
    dec = codecs.getincrementaldecoder("utf-8")()
    try:
        dec.decode(head)   # a partial char at the edge simply isn't flushed
        return False
    except UnicodeDecodeError:
        return True

That returns False for UTF-8 text no matter where the boundary lands, CJK included.

Wrapping up

I'm fairly sure this is the whole class of the bug, but I've only hit it on Windows, so your mileage may vary on a Linux box with a non-UTF-8 locale (same mechanism, much rarer default). If a tool prints a name that looks correct and still can't find it, hexdump the config and hexdump the value it's comparing against. The extra bytes will be sitting right there. Hope this saves somebody the afternoon it cost me.

Where I ran into it: langchain-ai/langgraph#8665 (config reads on a CJK locale), NousResearch/hermes-agent#97322 (the cp1252 pipe reader thread), NousResearch/hermes-agent#86187 (valid UTF-8 flagged as binary).

Two writers, one database: why your self-hosted app breaks the moment it runs twice

MilkyWay008 — Fri, 11 Sep 2026 17:09:24 +0000

I keep running into the same failure in the self-hosted world, and it took me an embarrassingly long time to name it the first time round. The app works fine as one container. You add a second replica, or a second backend process, and everything goes strange: logins stop sticking, requests hang, or the logs fill up with database is locked. Nothing crashes loudly. It just quietly stops making sense.

Two separate mistakes tend to get made at once when you run a second copy of something that was built as one instance, and both of them look like somebody else's bug.

Your old session dies, but a fresh one works

This is the one that fools people. You log in, the page works, you click something and you get bounced back to the login screen. Open an incognito window, log in again, and it behaves (at least for a while).

What's happening is that the instance that answered your login is not the instance answering your next request. If the app auto-generates a signing secret on first start, and Open WebUI calls this WEBUI_SECRET_KEY, each copy invents its own. Instance A signs your session cookie, instance B tries to verify it with a different key, and the honest answer it gives you is 401. A fresh login works because it gets minted and used by whichever pod you landed on.

On Kubernetes this sometimes arrives as an Envoy message instead of a clean 401: upstream connect error or disconnect/reset before headers. That's transport-level noise sitting on top of a state-level problem.

Or the logs say database is locked

The other half of this is a file that only tolerates one writer. SQLite allows exactly one writer at a time. If a writer can't take the lock it returns SQLITE_BUSY and the app reports database is locked, unless a busy handler is configured to wait. Write-ahead logging lets readers and writers share on the same machine, but the wal-index lives in an mmapped -shm file, so WAL does not work across machines, and the SQLite docs say it plainly: the WAL implementation will not work on a network filesystem. Rollback journals are no better, they use fcntl() locks that are broken on plenty of NFS implementations.

So the tempting fix, "put the .db on a shared volume and scale the replicas", isn't a fix at all. It's a faster route to corrupt data.

First, count your writers

Before changing anything, prove how many copies are running and who owns the state file.

# processes and pods
pgrep -af "serve|gateway|uvicorn"
docker ps --format '{{.Names}}\t{{.Ports}}'
kubectl get pods -o wide

# who holds the file and the port
lsof /path/to/webui.db          # or: fuser -v /path/to/webui.db
ss -ltnp | grep 8080

Two rows where you expected one is the whole diagnosis. Also look for stale PID or lock files in the app's data directory. A process that died badly leaves its lock behind, and the next start either refuses to boot or quietly becomes a second writer.

The fix, in order

Get back to one writer first. On macOS actually quit the app (Cmd+Q, not just closing the window), or run the app's own gateway stop, or kubectl scale deployment open-webui --replicas=1. Recovery before redesign.
Give every replica the same secret. One openssl rand -base64 32 value, injected into all pods as WEBUI_SECRET_KEY (and OAUTH_SESSION_TOKEN_ENCRYPTION_KEY if you use OAuth). This is the step people skip, because everything boots fine without it.
Move the database out of the container: DATABASE_URL=postgresql://user:pass@db-host:5432/openwebui. One warning the docs are clear about, Open WebUI does not migrate your existing SQLite data into Postgres for you. Do this before you have production data, or plan the export.
Externalize coordination as well: REDIS_URL=redis://redis-host:6379/0, plus WEBSOCKET_MANAGER=redis and ENABLE_WEBSOCKET_SUPPORT=true. Without Redis, websocket handling and config sync stay in-process, and multi-instance users get 403s and intermittent auth weirdness that looks nothing like a database problem.
Check the side databases too. The default ChromaDB vector store is SQLite-backed and not fork-safe, so vector search falling over on two replicas is the same disease in a different organ. Swapping to VECTOR_DB=pgvector puts it in the Postgres you already have.
Orchestrator hygiene: keep UVICORN_WORKERS=1 per container, and let exactly one replica run migrations (ENABLE_DB_MIGRATIONS=false on the rest).
Verify instead of hoping. Compare kubectl exec <pod> -- printenv WEBUI_SECRET_KEY across both pods, then log in and roll the deployment while keeping that session alive. Watch grep -c "database is locked" in the logs stop climbing.

Things that don't fix it

A shared NFS or PVC volume holding the SQLite file, with two replicas on top, doesn't fix anything. The Open WebUI docs warn you'll get database is locked and data corruption, and the SQLite WAL docs explain why the shared-memory file can't cross hosts anyway.

Sticky sessions don't fix it either...... sessionAffinity: ClientIP only hides the problem until a rollout moves your pod, and then everyone is logged out at once.

Copying a live .db around with docker cp or plain cp is worse than it looks. A copy taken mid-transaction is a mix of old and new pages, and if there's a -wal or -journal file sitting next to it, that has to travel with it or your copy is unusable. Use VACUUM INTO, the backup API, or sqlite3_rsync on 3.47 and newer.

Raising the busy timeout buys you nothing. No pragma makes SQLite safe on network storage.

The honest caveats

The Kubernetes issue I started from is still open, and the maintainers there haven't confirmed a root cause (fwiw, treat that particular attribution as community reasoning rather than gospel). The SQLite documentation is firm about network filesystems but never says the word Docker, so the bind-mount case is inference on my part, not a quote. And if your app ships a single-instance lease or lock, use it. The lease is the real fix, everything above is a workaround.

I could be wrong about the specifics of your stack, but if you're seeing session weirdness that only shows up once a second copy is running, this is the first place I'd look. It cost me a couple of hours of staring at logs the first time.

Sources worth reading: SQLite WAL, SQLite FAQ, locking, how to corrupt, and Open WebUI's scaling and hardening pages.