5 Ways to Quickly Run Self-Hosted Applications with Docker Compose

Recently, while prototyping a new side product on my own server, I looked for a faster way than manually spinning up services every time or dealing with complex deployment scripts. Docker Compose has always been a practical solution for me, especially for single-host environments, for quickly deploying and managing applications. In this post, I'll share five fundamental approaches I frequently use to run your self-hosted applications more efficiently and reliably with Docker Compose.

Docker Compose is a tool for defining and running multi-container Docker applications. It allows you to configure all your application's services, networks, and storage volumes from a single place using a YAML file. This way, you can spin up, stop, and manage applications with complex dependencies with a single command. This approach provides great convenience, especially for rapid prototyping, development environments, and small-scale production deployments.

1. Reduce Initial Setup Cost with Minimal Compose Files

When developing or deploying a self-hosted application, minimizing initial setup complexity is crucial. Often, all we need to get a service up and running is an image definition and perhaps a few port and volume settings. Trying to create a massive docker-compose.yaml file by thinking of every detail at first is both a waste of time and unnecessary fatigue. I usually start with the most basic requirements and iteratively develop the file as the application runs and needs become clearer.

For a typical web application consisting of a database, an API service, and a frontend, my initial docker-compose.yaml file usually looks like this:

version: '3.8'
services:
  database:
    image: postgres:15-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: myapp_db
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
    volumes:
      - db_data:/var/lib/postgresql/data
  api:
    build: ./api
    restart: unless-stopped
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgres://myuser:mypassword@database:5432/myapp_db
    depends_on:
      - database
  frontend:
    build: ./frontend
    restart: unless-stopped
    ports:
      - "3000:3000"
    depends_on:
      - api
volumes:
  db_data:

In this example, I've defined database, api, and frontend services. Each service has a basic image or build command, a restart policy, necessary environment variables, and ports. Specifying inter-service dependencies with depends_on ensures that services start in the correct order. The volumes definition ensures the persistence of database data. This simple approach allows me to quickly test the core functionalities of the application and avoid getting bogged down with unnecessary details in the initial iterations.

💡 Minimalism for Quick Testing

When running a service for the first time, start with only essential settings like image and ports. Once you've verified the basic functionality of your application, you can add more complex configurations like volumes, networks, or healthcheck. This also simplifies the debugging process.

This minimal structure is a method I frequently use when testing a new module for a production ERP or developing the backend for my own Android application. Especially when I need to quickly validate a concept, avoiding the overhead of complex YAML files helps me maintain project momentum. Instead of trying to perfect everything at first, starting with a working skeleton allows me to see the next steps more clearly.

2. Smartly Managing Environment Variables: .env and Compose Variables

When self-hosting my applications, I never want sensitive information (API keys, database passwords) to remain explicitly written within the docker-compose.yaml file. This poses both a security risk and makes managing configuration across different environments (development, test, production) difficult. This is where .env files and Docker Compose's variable management capabilities come into play. For me, this is one of the cornerstones of a flexible and secure deployment.

An .env file is a simple text file located in the same directory as your docker-compose.yaml file, containing key-value pairs:

# .env file
POSTGRES_PASSWORD=mysupersecretpassword
API_KEY=anothersecretkey
FRONTEND_PORT=3000

In the docker-compose.yaml file, I can reference these variables using $VARIABLE_NAME or ${VARIABLE_NAME}:

version: '3.8'
services:
  database:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: myapp_db
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} # Pulled from .env file
    volumes:
      - db_data:/var/lib/postgresql/data
  api:
    build: ./api
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgres://myuser:${POSTGRES_PASSWORD}@database:5432/myapp_db
      API_SECRET: ${API_KEY}
    depends_on:
      - database
  frontend:
    build: ./frontend
    ports:
      - "${FRONTEND_PORT}:${FRONTEND_PORT}" # Pulled from .env file
    depends_on:
      - api
volumes:
  db_data:

This way, I can separate sensitive information from the YAML file and exclude the .env file from version control systems like git (I usually add it to .gitignore). Furthermore, I can reuse the same docker-compose.yaml file across different environments by using different .env files. For example, I can create dev.env for the development environment and prod.env for production, and then spin up specific environments with commands like docker compose --env-file prod.env up -d.

⚠️ Security of Sensitive Information

.env files are a great way to separate sensitive information from version control, but make sure you store these files securely on the server (e.g., correct file permissions chmod 600 .env or use a better secret management solution). In production environments, it's generally safer to consider more advanced secret management systems like Kubernetes Secrets or HashiCorp Vault.

When working on a production ERP, managing database connection details or external service keys between test and production environments was always a challenge. .env files made these transitions seamless, allowing us to safely use the same Compose file for different environments. This way, we not only limited the concept of "working code" to application code but also included deployment configuration within this scope.

3. Correctly Using Volumes for Persistent Data

Containers are inherently ephemeral; that is, all data within them is lost when they are stopped or deleted. When I need persistent data in my self-hosted applications, such as databases, user uploads, or configuration files, using Docker Volumes is essential. Incorrect volume configuration can lead to serious problems, such as losing all data when a server restarts. Therefore, correctly configuring volumes is critical for application continuity and data security.

I use two main types of volumes in Docker Compose:

1. Named Volumes: These are volumes managed by Docker, usually visible with the docker volume ls command. They are the best way to store data independently of the container and are often preferred for critical data like databases.
   

   version: '3.8'
   services:
     database:
       image: postgres:15-alpine
       volumes:
         - db_data:/var/lib/postgresql/data # Named volume
   volumes:
     db_data: # Volume definition
   

2. Bind Mounts: These directly mount a specific directory from the host machine into the container. This is useful for instantly reflecting code changes during development or providing configuration files from the host to the container. However, for databases in production, Named Volumes are a safer and more manageable option.
   

   version: '3.8'
   services:
     api:
       build: ./api
       volumes:
         - ./api:/app # Binds host directory to container
   

When I use named volumes like db_data, my database data remains safe even if I stop the services with docker compose down. This provides great peace of mind, especially when I need to restart a service due to an update or an error.

ℹ️ Volume Backup

Using volumes for data persistence is important, but it doesn't eliminate the need for backups. You can regularly back up your named volumes using commands like docker cp or docker run --rm --volumes-from db_container -v $(pwd):/backup alpine tar cvf /backup/backup.tar /var/lib/postgresql/data.

I remember accidentally binding database data to a temporary directory on the host using a bind mount in a client project. All data was lost when the server restarted. This incident painfully taught me the difference between Named Volumes and Bind Mounts and when each should be used. Since then, I always make sure to use Named Volumes for critical data and even design my backup strategy with these volumes in mind.

4. Optimizing Network Configuration: Custom Networks and Port Mapping

When running multiple services with Docker Compose, inter-service communication needs to be both secure and efficient. By default, Docker Compose creates a single default network for your application and connects all services to it. However, when more isolation, security, or a complex network topology is required, defining custom networks and optimizing port mapping strategies becomes very useful.

For example, a frontend service should only be accessible by an Nginx reverse proxy, while a backend service should only be accessible from the frontend or another internal service. In this scenario, I can restrict inter-service access by defining different networks:

version: '3.8'
services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    networks:
      - external_network
      - internal_network
  frontend:
    build: ./frontend
    networks:
      - internal_network
  api:
    build: ./api
    networks:
      - internal_network
  database:
    image: postgres:15-alpine
    networks:
      - internal_network
    environment:
      POSTGRES_DB: myapp_db
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
    volumes:
      - db_data:/var/lib/postgresql/data

networks:
  external_network:
    driver: bridge
  internal_network:
    driver: bridge
volumes:
  db_data:

In this configuration, the nginx service is on both the external_network (internet-facing) and the internal_network (inter-application service communication). The frontend, api, and database services, however, only run on the internal_network. This prevents direct external access to the database service. Additionally, services can communicate with each other by their names (e.g., the api service can access the database service via database:5432), which simplifies configuration.

💡 Port Mapping Tips

When port mapping, we use the HOST_PORT:CONTAINER_PORT format. If you don't specify HOST_PORT (e.g., -"8000" instead of - "80"), Docker assigns a random host port, which you can find with docker compose port service_name 8000. However, it's usually desirable to use a specific port. To avoid conflicts in a development environment, you can change the HOST_PORT (e.g., - "8080:80").

When developing an internal platform for a bank, different services needed to operate in different security zones. By using Custom networks, we achieved strict isolation between these zones. By allowing only specific services to communicate with each other, we limited the spread of a potential security breach. This is a reflection of network segmentation and Zero-Trust architecture principles, not just in Docker Compose environments, but in general. Nginx being only externally exposed, while other services reside securely within their internal networks, is a fundamental step in such architectures.

5. Increasing Reliability with Health Checks and Restart Policies

A self-hosted application doesn't just need to be "running"; it also needs to be "healthy" and capable of automatically recovering from unexpected situations. Docker Compose allows us to increase this reliability with its healthcheck mechanism and restart policies. Understanding whether a service is truly operational and ensuring it automatically restarts when it crashes significantly reduces my operational burden.

A healthcheck definition allows Docker to periodically check the status of the application inside a container. For example, for a web service, we can check if it responds by making a request to a specific HTTP endpoint:

version: '3.8'
services:
  api:
    build: ./api
    ports:
      - "8000:8000"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 20s
    restart: unless-stopped
    depends_on:
      - database

In this example, I've defined a healthcheck for the api service. Docker will make a curl request to http://localhost:8000/health every 30 seconds. If it doesn't receive a response within 10 seconds or if an HTTP error code is returned (with curl's -f flag), it will be counted as a failure. After three failed attempts, the container will be marked as unhealthy. start_period allows sufficient time for the application to start; even if the health check fails during this period, a restart won't be triggered.

restart policies determine when a container will automatically restart:

• no: No automatic restart.
• on-failure: Restarts if the container exits with an error code (non-zero).
• always: The container is always restarted (until manually stopped).
• unless-stopped: The container is always restarted unless manually stopped (my most frequently used).

🔥 Misuse of Health Checks

A misconfigured healthcheck can cause your application to continuously restart (restart loop). For example, a service that performs a healthcheck before connecting to the database will continuously fail and restart if the database is not yet ready. In this case, it's important to correctly manage dependencies with start_period or depends_on.

I was using a Redis service in the backend of my own side product, and sometimes the Redis container would crash due to out-of-memory (OOM) issues. Thanks to the restart: unless-stopped policy, Redis would automatically restart, ensuring my application ran without interruption. However, adding a healthcheck to verify that Redis was truly operational ensured that the application could actually serve requests, beyond just the container being alive. This dual approach is an indispensable combination for increasing the resilience of my self-hosted systems.

User
  │
  ▼
Nginx Reverse Proxy
  │
  ▼
Frontend
  │
  ▼
API
  │
  ▼
PostgreSQL

Conclusion

Docker Compose is an invaluable tool for someone with field experience like me, for quickly and reliably spinning up self-hosted applications. Starting quickly with minimalist docker-compose.yaml files, smartly managing environment variables with .env, using Named Volumes for critical data, optimizing network configuration with Custom Networks, and finally increasing application resilience with Health Checks and Restart Policies are my frequently used core strategies.

These approaches not only run an application but also make it more manageable, secure, and flexible. Each step has been distilled from real-world problems and solutions I've encountered. I hope this guide helps you get the most out of Docker Compose in your own self-hosted projects and saves you from operational headaches. In my next post, perhaps I'll share my experiences on CI/CD integration with Docker Compose.