Stop Using .env Files for Docker Secrets — Try This Instead

#opensource #docker #kubernetes #devops

Managing secrets in Docker usually starts simple—a .env file here, a hardcoded variable there. But once you move past a single local machine, things get messy quickly.

Kubernetes users are spoiled for choice with tools like External Secrets Operator (ESO), but for those of us running standalone Docker or Compose on a handful of VMs, the options have always felt a bit "hacked together." You’re often stuck with the manual overhead of syncing files, or worse, hard-coding "temporary" credentials that inevitably end up in a Git commit.It works — until it doesn’t.

I started Docker Secret Operator (DSO) as a side project to solve a specific headache: I wanted my containers to talk to cloud secret managers without managing local files or SSHing into servers every time a password changed.

What is DSO?

DSO is a lightweight way to manage secrets in plain Docker environments. It consists of two parts: a background Agent that talks to your cloud provider (AWS, Azure, Vault, etc.) and a Docker CLI Plugin that lets you run containers with those secrets injected at runtime.

The goal was simple: your application code should just see an environment variable like DB_PASSWORD, but that password should never exist as a plain text string on your server’s disk.

Comparison: How do we handle secrets?

Feature	.env Files	Docker Secrets (Swarm)	DSO
Storage	Plaintext on disk	Encrypted (Swarm only)	In-memory (RAM)
Cloud Sync	Manual	No	Native (AWS/Vault/etc)
Rotation	Manual restart	Manual	Automatic

What Problem It Solves

If you aren't on Kubernetes, your options are usually a bit messy. You're either hoping nobody with read access to the server steals your .env files, or you're setting host environment variables manually, which makes automation a nightmare.

DSO fills this gap. It fetches secrets directly from your source of truth and injects them into your containers. If you update a secret in your cloud console, DSO can detect that change and automatically rotate the credentials in your running containers—no manual intervention needed.

How It Works

At a high level, it works like this:

DSO Agent: Runs as a systemd service. It authenticates with your provider, fetches secrets, and keeps them in an in-memory RAM cache. They never touch the physical disk.
CLI Plugin: You use docker dso up instead of docker compose up.
Communication: The CLI talks to the agent over a secure Unix domain socket (/var/run/dso.sock).
Injection: The plugin overlays the secrets onto your container's environment just as it starts.

Key Features

Zero-Persistence: Secrets live in RAM, never on disk.
Multi-Cloud: Native support for AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, and Huawei CSMS.
Auto-Rotation: Refreshes the cache and can trigger rolling restarts when a secret changes.
Native UX: It’s a Docker plugin, so it works seamlessly with the standard docker command.
Tiny Footprint: Written in Go with minimal resource usage.

Installation & Quick Setup

You can get up and running on Ubuntu or Debian with a single command:

curl -fsSL https://raw.githubusercontent.com/docker-secret-operator/dso/main/install.sh | sudo bash

Next, create your configuration file at /etc/dso/dso.yaml. Here’s what a typical AWS setup looks like:

providers:
  aws-prod:
    type: aws
    region: us-east-1    # Change to your AWS region
    auth:
      method: iam_role

agent:
  cache: true
  watch:
    polling_interval: 5m    

defaults:
  inject:
    type: env
  rotation:
    enabled: true
    strategy: rolling # Default strategy for all secrets

secrets:
  - name: arn:aws:secretsmanager:REGION:ACCOUNT:secret:YOUR_SECRET_NAME
    provider: aws-prod
    rotation:
      strategy: restart # Override global default for this specific secret
    mappings:
      DB_USER: MYSQL_USER
      DB_PASSWORD: MYSQL_PASSWORD

Start the agent:

sudo systemctl start dso-agent.service

Example Usage

In your docker-compose.yaml, you just reference the keys without values:

services:
  api:
    image: my-app:latest
    environment:
      - DB_USER      # DSO will fill this
      - DB_PASSWORD  # DSO will fill this

Now, instead of the standard compose command, run:

docker dso up -d

DSO will fetch the credentials from AWS, map them to your environment variables, and start your stack. No .env files to manage.

Why I Built This

I built this because I was tired of the "secret drift" problem. I'd update a database password in the cloud console, and then have to remember which five EC2 instances needed an .env update and a manual docker compose restart.

Missing even one instance usually led to a production fire at 2 AM. I wanted a "set it and forget it" solution—something that felt native to the Docker CLI but had the brains to handle fetching and rotating secrets automatically. DSO is the result of that frustration.