[Homelab AI Project] Ep.4 - Isolating the Service Layer with Docker Compose

Series: Building a Personalized AI News Pipeline (4/23)

Executive Summary

This post details how to design a multi-service container infrastructure on a single homelab server (Ryzen 5500U, 16GB RAM) using Docker Compose. We implement hardware resource caps via cgroups (allocating 4GB to PostgreSQL and 1GB to Redis) and establish a dual-network topology (Internal vs. External) to prevent resource contention and secure data stores. We also share troubleshooting lessons on enabling Linux memory controllers, Docker DNS search ordering, and environment parser edge cases.

References: docker-compose resource limits docs / Linux cgroups documentation

---

1. The Challenge: Monolithic Mess vs. Virtualization Overhead

Our AI News Pipeline consists of four distinct architectural components:

1. RSS Collector (Node.js): Constantly crawls tech feeds.
2. AI Processing Engine (Python): Handles embedding generation, Vector DB querying (RAG), and LLM orchestration.
3. Database (PostgreSQL + pgvector): Stores articles, metadata, and high-dimensional vectors.
4. Queue & Cache (Redis 7.2 Alpine): Orchestrates tasks and caches API requests.

In a naive setup, one might install these services directly on the host OS (dnf install postgresql-server redis). However, in a production-grade setting, this leads to:

• Resource Contention: Heavy embedding computation in Python can spike CPU and memory, triggering the host OS's Out-Of-Memory (OOM) Killer to terminate PostgreSQL.
• Dependency Conflicts: Upgrades to system libraries for one service can break dependencies for another.

To avoid this, we could use hypervisor-level virtualization (such as Proxmox or ESXi) to allocate resources statically. But on a mini PC with only 16GB of RAM, running multiple VM operating systems introduces unacceptable virtualization memory overhead.

The optimal solution is containerization with Docker Compose -- providing lightweight process isolation, strict resource control, and seamless portability from local environment to our production Rocky Linux server.

---

2. The Architecture: Cgroups and Network Segmentation

To adapt enterprise-grade network security and resource scheduling principles to our homelab, we implemented three key design patterns:

1. Resource Constraints via cgroups

Docker utilizes Linux kernel cgroups to restrict container workloads. We enforced hard limits to protect host OS stability:

• PostgreSQL (ainews-db): Limit to 4GB RAM, 2.0 CPUs. Enough for pgvector indexing and cache buffers, but capped to prevent host OS starvation.
• Redis (ainews-queue): Limit to 1GB RAM, 1.0 CPU. Activates eviction policies if memory usage exceeds this cap.

2. Dual-Network Partitioning

Containers should not share a single flat network. We separated traffic into two custom bridge networks:

• ainews-internal: Only contains PostgreSQL (ainews-db) and Redis (ainews-queue). It has no outbound internet route (internal: true) and is inaccessible from the outside world.
• ainews-external: Contains the public-facing Frontend (Astro), the Collector (which needs outbound internet access to fetch RSS feeds), and the Cloudflare Tunnel agent (cloudflared).

3. Complete Configuration Decoupling

All secrets, ports, and volume paths are abstracted into a local .env file, allowing us to keep configuration separate from the core stack declaration.

graph TD
    subgraph Host_OS ["Rocky Linux 10.1 (Host)"]
        subgraph Ext_Net ["ainews-external Network"]
            CF_Tunnel["Cloudflare Tunnel (cloudflared)"]
            Astro_FE["Astro + React Frontend"]
            Collector["Node.js RSS Collector"]
            AI_Engine["Python AI Engine"]
        end

        subgraph Int_Net ["ainews-internal Network"]
            Postgres["PostgreSQL + pgvector (ainews-db) <br> [cgroups limit: 4GB]"]
            Redis["Redis 7.2 Alpine (ainews-queue) <br> [cgroups limit: 1GB]"]
        end
    end

    CF_Tunnel --- Astro_FE
    Astro_FE --- Postgres
    Collector --- Postgres
    Collector --- Redis
    AI_Engine --- Postgres
    AI_Engine --- Redis

    style Postgres fill:#336791,stroke:#fff,stroke-width:2px,color:#fff
    style Redis fill:#D82C20,stroke:#fff,stroke-width:2px,color:#fff
    style CF_Tunnel fill:#F38020,stroke:#fff,stroke-width:2px,color:#fff
    style Ext_Net fill:#e1f5fe,stroke:#0288d1,stroke-width:1px,stroke-dasharray: 5 5
    style Int_Net fill:#efebe9,stroke:#5d4037,stroke-width:1px,stroke-dasharray: 5 5

---

3. Setup Files: infra/docker-compose.yml & .env.example

infra/.env.example

PROJECT_NAME=ai-news

POSTGRES_USER=tristan
POSTGRES_PASSWORD=super-secret-password-change-me
POSTGRES_DB=ainews
POSTGRES_PORT=5432
DB_VOLUME_PATH=./volumes/postgres

REDIS_PORT=6379
REDIS_VOLUME_PATH=./volumes/redis

NODE_ENV=production

infra/docker-compose.yml

version: '3.8'

networks:
  ainews-internal:
    driver: bridge
    internal: true
  ainews-external:
    driver: bridge

services:
  # 1. PostgreSQL DB with pgvector
  ainews-db:
    image: postgres:16
    container_name: ainews-db
    restart: always
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - ${DB_VOLUME_PATH}:/var/lib/postgresql/data:z # :z is required for SELinux context
    ports:
      - "${POSTGRES_PORT}:5432"
    networks:
      - ainews-internal
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 4096M

  # 2. Redis Cache & Queue
  ainews-queue:
    image: redis:7.2-alpine
    container_name: ainews-queue
    restart: always
    command: redis-server --appendonly yes --requirepass ${POSTGRES_PASSWORD}
    volumes:
      - ${REDIS_VOLUME_PATH}:/data:z
    ports:
      - "${REDIS_PORT}:6379"
    networks:
      - ainews-internal
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1024M

  # 3. Node.js RSS Collector
  ainews-collector:
    image: node:22-slim
    container_name: ainews-collector
    restart: always
    working_dir: /usr/src/app
    environment:
      NODE_ENV: ${NODE_ENV}
      DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@ainews-db:5432/${POSTGRES_DB}      REDIS_URL: redis://:${POSTGRES_PASSWORD}@ainews-queue:6379/0    networks:
      - ainews-internal # Primary network first to prioritize internal DNS resolution
      - ainews-external
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 512M

---

4. Real-World Troubleshooting

Issue 1: Cgroup Memory Limits Ignored under Standalone Compose

After starting the stack, running docker stats revealed that resource caps were not being enforced; the memory limits showed the host PC's full memory allocation (16GB).

• Cause: On certain installations of modern RHEL/Rocky Linux environments running cgroups v2, memory controllers may not automatically map limits down to containers unless the system boot parameters explicitly delegate memory swapping and namespace control.
• Resolution: Update the GRUB kernel parameters using grubby to ensure the memory cgroup controller is active at boot:

  # Check if memory controller is enabled in cgroups
  cat /proc/cgroups | grep memory

  # If disabled, enable memory and swap accounting via grubby
  sudo grubby --update-kernel=ALL --args="cgroup_enable=memory swapaccount=1"
  sudo reboot

After restarting, docker stats correctly recognized the limits (4GiB for DB, 1GiB for queue).

Issue 2: DNS Resolution Failures inside Dual-Network Containers

The collector container crashed immediately on startup, failing to connect to the DB. The logs outputted:
Error: getaddrinfo ENOTFOUND ainews-db

• Cause: When a container is attached to multiple bridge networks (in this case, both ainews-internal and ainews-external), Docker's internal DNS resolver configures the search path based on the sequence in which networks are declared. If the external network is registered first, DNS requests for internal hosts (like ainews-db) may get routing priority on the external network gateway, failing to resolve the internal hostname.
• Resolution: Reorder the networks block under the service definition in docker-compose.yml, placing the private network (ainews-internal) at the top of the array:

  networks:
    - ainews-internal  # Prioritize private network DNS
    - ainews-external

This forces the container's resolver to query the internal bridge first, resolving internal hostnames instantaneously.

Issue 3: Docker Compose .env Parsing Failure with Secret Special Characters

The database container failed to launch, reporting authentication syntax errors even though the password matched our .env configuration.

• Cause: We generated a strong password containing a # character. The standard Docker Compose environment parser reads .env lines literally and interprets # as the beginning of a comment. Consequently, everything after the # character was stripped out, leaving PostgreSQL with a truncated password.
• Resolution: Wrap any variable values containing special characters in double quotes (") inside the .env file to escape the parser:

  POSTGRES_PASSWORD="my#secret$password"

---

5. Results & Metrics

Here are the resource usage statistics captured under active crawling load:

Container	Memory Limit	Memory (Idle)	Memory (Peak)	CPU Limit	CPU (Peak Load)
ainews-db	4096 MiB	~120 MiB	1.8 GiB (RAG Queries)	2.0 Cores	~45%
ainews-queue	1024 MiB	~12 MiB	~85 MiB (High Queue Load)	1.0 Core	~5%
ainews-collector	512 MiB	~65 MiB	~180 MiB (Parallel Parsing)	1.0 Core	~35%

• Zero Resource Contention: No matter how computationally intensive our RAG indexing operations get, ainews-db is strictly locked out of hogging the entire host memory, preserving a safe overhead of 1.5GB for host OS systemd tasks and the Cloudflare Tunnel daemon.
• Port Scans Mitigated: Because our public-facing ports are bound internally and exposed strictly via cloudflared edge routing, external port scanning attempts against our home IP register a complete drop (closed status), keeping server audit logs clean of noise.

---

6. Next Up

With our backend services isolated and secured in Docker Compose, the next challenge is managing deployments. In Episode 5, we will set up Gitea (a lightweight 100MB self-hosted Git server) and Act Runner to configure a seamless, local CI/CD pipeline on push events.