Try to open a DuckDB file from two processes at once and you get this:
_duckdb.IOException: IO Error: Could not set lock on file "server.duckdb":
Conflicting lock is held in /usr/bin/python3.11 (PID 1355).
That has been true since the beginning. DuckDB is an in-process database: one file, one writer, full stop. DuckDB 2.0, which went GA on 21 October 2026, ships an extension called quack that turns a running DuckDB process into a small network server other DuckDB processes can connect to. I wanted to know whether that actually gets you past the lock, what it costs per query, and what it still refuses to do.
I ran DuckDB 2.0.0 through the Python client (pip install duckdb==2.0.0), started a server on a table with a million rows, and hit it from separate Python processes acting as independent clients.
Setting it up
The server side loads the extension and calls quack_serve:
con = duckdb.connect('server.duckdb')
con.execute("INSTALL quack; LOAD quack;")
con.execute("CREATE TABLE t AS SELECT range AS i, 'row_'||range AS s FROM range(1000000)")
con.execute("CALL quack_serve('quack:localhost:9494', token='testtoken123')")
A client attaches to it as a remote database and routes its whole session there with CONNECT:
con = duckdb.connect()
con.execute("INSTALL quack; LOAD quack;")
con.execute("ATTACH 'quack:localhost:9494' AS srv (TOKEN 'testtoken123')")
con.execute("CONNECT srv")
con.execute("SELECT count(*) FROM t").fetchall() # [(1000000,)]
con.execute("DISCONNECT")
CONNECT takes about 3ms the first time and the client then talks to the server for every query until it disconnects.
The lock is still there, and that's the point
Opening server.duckdb directly from a second process while the server holds it still fails with the exact error above. Quack doesn't touch DuckDB's file-locking model at all. What it does is give you a second door into the one process that is allowed to hold the write lock, so that ten processes can write through that one door instead of fighting over the file.
I also checked whether the lock problem was ever really about reads. It isn't: with no writer active, two processes can open the same file with read_only=True at the same time and both query it fine. That already worked before 2.0. Quack's actual contribution is concurrent writes, not concurrent reads, which is a narrower claim than "multiple processes can now use one DuckDB file" might suggest.
Writes that used to be impossible
I ran 10 client processes, each opening its own connection through CONNECT and firing 100 single-row inserts, 1,000 inserts in total.
| Setup | Errors | Rows landed | Distinct (client, seq) pairs |
|---|---|---|---|
| 2 processes, direct file open | 1 (lock error, immediate) | 0 | — |
10 clients via quack CONNECT
|
0 | 1,000 | 1,000 |
Every insert landed, nothing was lost or duplicated, and the server-side count matched the client-side count exactly across three repeats of this test. That is the headline: a workload that used to be impossible to run from more than one process at a time now runs correctly from ten.
Concurrency doesn't scale the way I expected
I measured throughput for the same 1,000 inserts split across 1, 5, 10 and 20 concurrent clients, three full sweeps:
| Clients | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| 1 (serial) | 401/s | 380/s | 348/s |
| 5 | 692/s | 735/s | 655/s |
| 10 | 542/s | 570/s | 508/s |
| 20 | 362/s | 332/s | 342/s |
Throughput peaks at 5 concurrent clients, then falls off. At 20 clients it drops back to roughly the same number as a single client running everything serially, sometimes below it. I didn't expect that, so I reran the whole sweep twice more to make sure it wasn't a fluke. It wasn't. DuckDB still executes writes against one file one at a time internally; quack's connection handling adds its own overhead on top, and past a handful of clients that overhead eats whatever the server-side queueing gains you. If your workload is many short-lived writers, five seems to be roughly where this particular setup stopped paying off and started costing more than it saved.
What a query costs going through the wire
Even on loopback, going through the server instead of talking to the file in-process adds a tax on every query, measured over 200 runs:
| Query | In-process | Via quack CONNECT
|
|---|---|---|
SELECT 1 (median) |
0.25ms | 1.92ms |
SELECT sum(i), count(*) FROM t (median, 1M rows) |
0.77ms | 2.42ms |
SELECT * FROM t (full 1M rows fetched) |
0.334s | 0.391s |
For small queries the absolute numbers are tiny but the ratio is large, about 7–8x for a trivial query. For a query that returns a lot of data, the gap narrows to roughly 17%, which is closer to what you'd expect from serialising a result set over HTTP instead of handing back memory directly.
Your local tables disappear while you're connected
This one surprised me. I created a local table, then ran CONNECT to the server:
CatalogException: Catalog Error: Table with name local_only does not exist!
Did you mean "pg_collation"?
I tried qualifying it explicitly with the schema name DuckDB normally uses for the default database:
CatalogException: Catalog Error: Table with name "memory.local_only" does not
exist because schema "memory" does not exist.
Both failed. While CONNECT-ed, the local catalog isn't just deprioritised, it's gone from view entirely, and there's no prefix that gets it back. DISCONNECT returns it immediately. If your plan was to pull a remote aggregate and join it against a local staging table in the same session, that plan doesn't work; you have to disconnect, or keep the remote session in a second connection object.
What it refuses, in its own words
A wrong token fails immediately at ATTACH, before CONNECT is even called:
_duckdb.InvalidInputException: Invalid Input Error: Authentication failed
Trying to bind the server to anything other than localhost without opting in fails the same way:
Invalid Input Error: Only localhost is allowed as a Quack RPC hostname by
default, set allow_other_hostname=true to override. We strongly recommend
reverse-proxying the Quack RPC when making it publicly available.
Both of those matched what the docs say, which was reassuring given how much else I had to work out by trial and error.
Watching it from the server side
quack_active_connections() sounds like a live connection list. It's closer to a short query log: it kept rows for finished queries, with status active or finished, the query text, a connection id and a timestamp, even after those queries completed. Useful for seeing what clients just ran, less useful as a "who is connected right now" view, since finished work stays visible until something ages it out.
What it costs to run
The server process holding the million-row table idled at about 120MB RSS. Under ten concurrent clients hammering it with small queries, that rose to roughly 132MB and settled back down to about 125MB once the load finished. For a table this size, the server itself isn't the expensive part.
One more thing worth knowing: killing the server process with kill -9 released the file lock immediately. A fresh direct connection to the same file afterwards worked on the first try, with all million rows intact. No stale lock file, no recovery step.
What I got wrong on the way
My first attempt at the client side used ATTACH 'quack://localhost:9494' AS srv, modelled on an ordinary URL. DuckDB took that literally as a filesystem path and failed with "Cannot open file", which looked like a connection problem and sent me checking the server log for several minutes before I noticed the double slash. The actual scheme has no slashes: quack:localhost:9494, the same shape DuckDB already uses for postgres: and mysql: connection strings. Once I matched that, everything else worked first time.
Run it yourself
python3 -m venv venv && ./venv/bin/pip install duckdb==2.0.0
Server:
import duckdb, time
con = duckdb.connect('server.duckdb')
con.execute("INSTALL quack; LOAD quack;")
con.execute("CREATE TABLE IF NOT EXISTS t AS SELECT range AS i, 'row_'||range AS s FROM range(1000000)")
print(con.execute("CALL quack_serve('quack:localhost:9494', token='testtoken123')").fetchall())
while True:
time.sleep(1)
Concurrent writers, run while the server above is up:
import duckdb, threading, time
def make_client():
con = duckdb.connect()
con.execute("INSTALL quack; LOAD quack;")
con.execute("ATTACH 'quack:localhost:9494' AS srv (TOKEN 'testtoken123')")
con.execute("CONNECT srv")
return con
setup = make_client()
setup.execute("CREATE TABLE IF NOT EXISTS writes (client_id INTEGER, seq INTEGER)")
setup.execute("DISCONNECT")
def worker(client_id):
con = make_client()
for seq in range(100):
con.execute(f"INSERT INTO writes VALUES ({client_id}, {seq})")
con.execute("DISCONNECT")
t0 = time.perf_counter()
threads = [threading.Thread(target=worker, args=(i,)) for i in range(10)]
for th in threads: th.start()
for th in threads: th.join()
print("elapsed", time.perf_counter() - t0)
What I'd do with this
If you have a single-node analytics workload where more than one process occasionally needs to write to the same DuckDB file, quack solves a real problem that had no good answer before. I wouldn't reach for it to serve many short, latency-sensitive clients: the per-query tax is real, and my own numbers show throughput getting worse, not better, past a handful of concurrent writers. I'd also budget for the catalog isolation explicitly, since it changes how you'd structure a session that needs both local and remote tables. Test your own concurrency sweet spot before picking a client pool size; mine was 5, and I have no reason to think that number generalises to a different machine or workload.
Top comments (0)