DEV Community

Libme
Libme

Posted on

Postgres "could not resize shared memory segment: No space left on device" in Docker: Why It Happens and How to Fix It

Short answer: this error means a parallel query tried to allocate more shared memory than the container's /dev/shm allows. Docker gives every container a 64MB /dev/shm by default, regardless of how much RAM the host has. Raise it with shm_size (Docker/Compose) or a memory-backed emptyDir mounted at /dev/shm (Kubernetes); disabling parallelism is a stopgap, not a fix.

What does the error look like?

ERROR:  could not resize shared memory segment "/PostgreSQL.1672401134" to 67146816 bytes: No space left on device
Enter fullscreen mode Exit fullscreen mode

The confusing part is that the disk is not full. df -h on your volumes looks fine, the container has gigabytes of memory headroom, and the same query works on your laptop's native Postgres install. It also tends to be intermittent: small queries pass, a bigger report query or a larger dataset after a data load triggers it.

The "No space left on device" refers to /dev/shm, a tmpfs. Check it from inside the container:

docker exec -it my-postgres df -h /dev/shm
# Filesystem      Size  Used Avail Use% Mounted on
# shm              64M     0   64M   0% /dev/shm
Enter fullscreen mode Exit fullscreen mode

If you see 64M, that is your ceiling.

Why does a parallel query need /dev/shm?

When the planner chooses a parallel plan, the leader process and its workers need to share data: tuple queues, and for a Parallel Hash join, the hash table itself. Postgres allocates that as dynamic shared memory. On Linux the default dynamic_shared_memory_type is posix, which creates segments named /PostgreSQL.<number> in /dev/shm.

You can confirm a query is a candidate by looking for parallel nodes:

EXPLAIN (ANALYZE, BUFFERS)
SELECT o.customer_id, count(*)
FROM orders o
JOIN order_items i ON i.order_id = o.id
GROUP BY o.customer_id;
-- Look for: Gather, Workers Planned: 2, Parallel Hash Join
Enter fullscreen mode Exit fullscreen mode

A Parallel Hash builds one shared hash table sized by work_mem multiplied by the number of participants (as of Postgres 13+ the shared hash table can use up to work_mem × hash_mem_multiplier per participant). With work_mem = 64MB and two workers plus the leader, one query can ask for hundreds of megabytes of shared memory. Against a 64MB tmpfs, that fails.

Takeaway: the error is a tmpfs size limit that only parallel plans with large hash tables hit, which is why it shows up after data growth rather than on day one.

What are the fixes, and which one should I use?

Fix What it does Cost / drawback
Raise /dev/shm size Gives parallel queries the memory they were planned for Shared memory counts toward container memory in some setups; size it deliberately
max_parallel_workers_per_gather = 0 Stops parallel plans entirely Slower large scans and aggregations; hides the problem
Lower work_mem Smaller hash tables, fewer shm bytes Can push sorts and hashes to disk, slowing other queries
dynamic_shared_memory_type = mmap Backs segments with files under pg_dynshmem instead of /dev/shm Disk-backed, so slower, and not recommended for production by the Postgres docs

For almost everyone, the first row is the right answer.

Docker and Compose

services:
  db:
    image: postgres:16
    shm_size: 256mb
    environment:
      POSTGRES_PASSWORD: example
Enter fullscreen mode Exit fullscreen mode

With plain docker run, the flag is --shm-size=256m. Recreate the container (a restart is not enough if you changed the Compose definition; run docker compose up -d) and re-check df -h /dev/shm.

How big? Start from your worst parallel query: roughly (workers + 1) × work_mem × hash_mem_multiplier is a reasonable upper bound for one hash join, and concurrent queries add up. I would rather size it to a few hundred MB and watch than set it to something huge, because tmpfs pages are real memory once written.

Kubernetes

Kubernetes has no shm_size field. The standard approach is an in-memory emptyDir mounted over /dev/shm:

spec:
  containers:
    - name: postgres
      image: postgres:16
      resources:
        limits:
          memory: 2Gi
      volumeMounts:
        - name: dshm
          mountPath: /dev/shm
  volumes:
    - name: dshm
      emptyDir:
        medium: Memory
        sizeLimit: 512Mi
Enter fullscreen mode Exit fullscreen mode

The thing that tripped me up here: memory-backed emptyDir usage is charged against the container's memory limit. If shared memory fills while shared_buffers and backend memory are already near the limit, you swap a "No space left on device" error for an OOMKill. Leave headroom in limits.memory beyond what you used to give the pod.

If you run Postgres through an operator such as CloudNativePG or Zalando's postgres-operator, check whether it already mounts /dev/shm for you before adding your own volume; operators commonly handle this and a duplicate mount will conflict.

Takeaway: raise /dev/shm to a few hundred MB and treat that memory as part of the container's budget, not free space.

How do I verify the fix?

Re-run the failing query and confirm the plan is still parallel, so you did not "fix" it by silently turning parallelism off:

docker exec -it my-postgres psql -U postgres -c "SHOW dynamic_shared_memory_type;"
docker exec -it my-postgres df -h /dev/shm
Enter fullscreen mode Exit fullscreen mode

Then run EXPLAIN (ANALYZE) and check that Workers Launched matches Workers Planned. If launched is lower than planned, the cluster is hitting max_parallel_workers, which is a different limit.

Do managed databases have this problem?

Not in a way you control. On managed offerings such as Amazon RDS for PostgreSQL, shared memory is part of the instance configuration, so you will not edit /dev/shm yourself. If you want to avoid owning this kind of container-level detail entirely, Amazon RDS for PostgreSQL is one option that removes it; the tradeoff is less control over kernel-level settings and a bill that scales with instance class. For a local dev database or a small self-managed deployment, the fix above is a one-line change and not a reason to migrate.

FAQ

Why does Postgres say "No space left on device" when my disk is empty?
Because the error refers to /dev/shm, an in-memory tmpfs, not your data volume. Docker defaults it to 64MB.

What should I set shm_size to for Postgres in Docker?
256MB is a sensible starting point for development and modest workloads. Size up if your parallel queries use large work_mem values, and verify with df -h /dev/shm during a heavy query.

Is it safe to set max_parallel_workers_per_gather = 0 to make the error go away?
It stops the error, but it also disables parallel query for that setting's scope, so large scans and aggregates will run slower. Use it to unblock yourself, then raise /dev/shm and turn parallelism back on.

Bottom line

If you run Postgres in a container and see "could not resize shared memory segment," set shm_size in Docker or mount a memory-backed emptyDir at /dev/shm in Kubernetes, and budget that memory inside your limits. Use max_parallel_workers_per_gather = 0 only as a temporary workaround, and avoid mmap shared memory for production. If you would rather not manage container internals at all, a managed Postgres service removes this class of problem at the cost of control.

Related reading

Top comments (0)