Raphael Gab-Momoh

Posted on Sep 17 Originally published at raphaelgmomoh.pages.dev

Rootless Containers: Converting a Real Image and Fixing What Breaks

#docker #security #rootlesscontainers #azurecontainerinstances

Why Most Images Still Run as Root

Docker Bench for Security, CIS benchmarks, and every container security checklist say the same thing: don't run your container process as root. Almost every tutorial Dockerfile does it anyway, because the fix looks like adding one line — USER node — and stopping there works for the simplest possible app. It doesn't work the moment the app touches the filesystem, and most articles about rootless containers never show you that failure, because they never tried to reproduce it.

This article takes the multi-stage app from the first article in this series, adds one small feature that writes a file at startup, converts it to run as a non-root user, and lets it actually crash — with the real error, not a description of one — before showing the fix.

Before Step 1, one term this walkthrough leans on:

The node user — the official node Docker images ship a pre-created, non-root user literally named node (UID 1000), specifically so you don't have to create your own with useradd. USER node in a Dockerfile switches every subsequent instruction, and the final container process, to run as that user instead of root — but it does nothing about file ownership for anything already copied into the image before that line runs.

Step 1 — Giving the app a reason to fail

import express from "express";
import fs from "fs";
import path from "path";

const app = express();
const PORT = process.env.PORT || 3000;
const DATA_DIR = path.join(process.cwd(), "data");
const STARTUP_LOG = path.join(DATA_DIR, "started.txt");

app.get("/healthz", (req, res) => {
  res.json({ status: "ok" });
});

app.listen(PORT, () => {
  fs.mkdirSync(DATA_DIR, { recursive: true });
  fs.writeFileSync(STARTUP_LOG, `started at ${new Date().toISOString()}\n`, { flag: "a" });
  console.log(`listening on ${PORT}`);
});

Why this one change matters for the article: the previous two articles in this series used an app that never wrote to the filesystem at runtime, which is exactly why a naive USER line would have worked fine on them — and would have taught nothing. Real applications write logs, cache files, or uploaded content somewhere. fs.mkdirSync on startup is the smallest possible version of that.

Step 2 — The naive conversion

FROM node:22-alpine AS build
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build

FROM node:22-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY package.json ./
RUN npm install --omit=dev
COPY --from=build /app/dist ./dist
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

Why this looks correct and isn't: every COPY and RUN instruction before USER node executes as root by default — that's Docker's own default, not something this Dockerfile opted into. WORKDIR /app creates /app owned by root. COPY --from=build /app/dist ./dist copies files in as root. The USER node line switches who runs the container process, but it has no effect on files and directories that already exist with root ownership from the layers built before it.

Step 3 — Running it for real and reading the actual crash

No local Docker daemon was available for this either, so the built image was run on Azure Container Instances instead — pull it, run it, read the logs, same idea as running it locally:

az acr build --registry $ACR_NAME --image rootless-lab:broken -f Dockerfile.rootless-broken .

TOKEN=$(az acr login --name $ACR_NAME --expose-token --output tsv --query accessToken)
az container create --resource-group $RG --name rootless-broken-test \
  --image $ACR_NAME.azurecr.io/rootless-lab:broken \
  --registry-login-server $ACR_NAME.azurecr.io \
  --registry-username 00000000-0000-0000-0000-000000000000 \
  --registry-password "$TOKEN" \
  --cpu 1 --memory 1 --restart-policy Never

az container logs --resource-group $RG --name rootless-broken-test

node:fs:1370
  const result = binding.mkdir(
                         ^

Error: EACCES: permission denied, mkdir '/app/data'
    at Object.mkdirSync (node:fs:1370:26)
    at Server.<anonymous> (file:///app/dist/server.js:12:8)
    ...
  errno: -13,
  code: 'EACCES',
  syscall: 'mkdir',
  path: '/app/data'
}

The container group's own state confirmed it: "state": "Failed". This is the exact failure mode that makes teams either give up on rootless containers or — worse — quietly add USER root back in a "temporary" fix that never gets reverted. Neither is necessary once the actual cause is visible.

Step 4 — The real fix: own the directory before switching users

FROM node:22-alpine AS build
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
RUN npm run build

FROM node:22-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY package.json ./
RUN npm install --omit=dev
COPY --from=build /app/dist ./dist
RUN mkdir -p /app/data && chown -R node:node /app
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

Why chown -R node:node /app has to happen before USER node, not after: chown itself requires root privileges to change ownership to a different user — running it as node would fail for the same reason the app's own mkdir failed. The ordering matters: root creates and pre-provisions the directory the app will need, hands ownership to node, and only then steps aside.

Step 5 — Verifying the fix, including who's actually running

az acr build --registry $ACR_NAME --image rootless-lab:fixed -f Dockerfile.rootless-fixed .
# ...create container the same way, pointing at :fixed...

az container logs --resource-group $RG --name rootless-fixed-test
# listening on 3000

curl http://<container-ip>:3000/healthz
# {"status":"ok"}

az container exec --resource-group $RG --name rootless-fixed-test --exec-command "id"
# uid=1000(node) gid=1000(node)

Three separate pieces of evidence, not one: the container's own logs show it started without crashing, curl proves the HTTP server actually answers, and az container exec ... id proves the process inside is genuinely running as UID 1000, not root — the actual property this whole exercise was for, confirmed rather than assumed from reading the Dockerfile.

Closing Thoughts

The gap between "add USER node" and "actually run rootless without breaking" is exactly one chown — but that gap is invisible until something inside the container tries to write to a directory it doesn't own, and most demo apps never do. The real lesson generalizes past this one fs.mkdirSync call: any container that logs to a file, caches to disk, or accepts uploads needs its writable directories explicitly handed to the non-root user before the app ever starts, and the only way to know you've done that correctly is to run it and watch it not crash — not to read the Dockerfile and assume it's fine.

GitHub Repository: docker-multistage-lab — includes Dockerfile.rootless-broken (reproduces the real EACCES crash) and Dockerfile.rootless-fixed, both runnable exactly as shown above.

Reviewed against current Azure CLI (az acr build, az container create, az container exec) as of September 2026.

Docker · Rootless Containers · Container Security · Azure Container Instances

Originally published on my portfolio.

DEV Community