DEV Community: Alex Miller

Remote Server Monitoring over VPN: A Docker Approach (Part 2)

Alex Miller — Thu, 23 Apr 2026 21:34:45 +0000

Recap: Where We Left Off

In Part 1 of this series, we established our secure foundation: an encrypted L3 bridge using AmneziaWG and Docker's Network Namespace sharing (network_mode). We already have our first agent, node-exporter, silently listening inside the tunnel on the remote node at 10.10.0.2:9100.

The traffic is encrypted, the ports are invisible to the public internet, and the whole thing is defined as code. But metrics sitting on a remote node are useless if nothing collects and visualizes them.

The Plan for Part 2

We still have three pieces missing from the stack. In this part, we will:

Add cAdvisor to remote nodes to monitor individual containers.
Configure Prometheus on the Hub to pull metrics through the tunnel.
Deploy Grafana to visualize everything in one place.

By the end, you will have a fully functional monitoring stack where the entire scrape path runs inside the encrypted tunnel, with no metrics endpoints reachable from the public internet.

Step 1: Monitoring Containers on Remote Nodes

While node-exporter gives us the "big picture" of the hardware (CPU, RAM, Disk), we need cAdvisor to see what's happening inside our containers.

Extend the remote node's docker-compose.yaml with a new cadvisor service. Just like the node-exporter, it must share the network stack of the VPN client — so no ports: block is needed, and the metrics endpoint will only be reachable through the tunnel.

services:
  # ... (wg-client and node-exporter services from Part 1) ...

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    restart: unless-stopped
    network_mode: "service:wg-client" # Shares the VPN tunnel
    depends_on:
      - wg-client
    devices:
      - /dev/kmsg
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
    command:
      - '--housekeeping_interval=30s'

Now your remote node is serving hardware metrics on port 9100 and container metrics on port 8080, but only via the private VPN IP (e.g., 10.10.0.2).

Step 2: Configuring the Scraper on the Hub

The core of our monitoring is Prometheus. Since we will run it inside the VPN container's network stack on the Hub, it can reach remote targets by their static tunnel IPs as if they were on a local LAN.

Create a file named ./prometheus/prometheus.yml on your Hub:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'remote-nodes-hw'
    static_configs:
      - targets: ['10.10.0.2:9100'] # Remote server: Node Exporter

  - job_name: 'remote-nodes-containers'
    static_configs:
      - targets: ['10.10.0.2:8080'] # Remote server: cAdvisor

  - job_name: 'hub-local'
    static_configs:
      - targets: ['localhost:9090']

Notice the target addresses: from Prometheus's point of view, 10.10.0.2 is just a regular LAN host. It does not know, and does not care, that packets are being encrypted and shipped across the public internet.

Step 3: Deploying Prometheus and Grafana on the Hub

Now let's update the Hub's docker-compose.yaml. We need to add Prometheus and Grafana, and extend the wg-monitoring service from Part 1 to publish the Prometheus UI port.

Note the networking logic: Prometheus sits inside the VPN network to "see" the remote targets. Grafana sits outside on a standard bridge network and talks to Prometheus through the VPN container's name.

services:
  wg-monitoring:
    # ... (base config from Part 1) ...
    ports:
      - "51820:51820/udp"   # VPN listener (from Part 1)
      - "9090:9090"         # Prometheus UI (see note below)

  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    network_mode: "service:wg-monitoring" # Hidden inside VPN namespace
    depends_on:
      - wg-monitoring
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - ./prometheus_data:/prometheus
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"         # Grafana UI (LAN access only)
    networks:
      - monitoring          # Talks to wg-monitoring via bridge
    environment:
      - GF_PROMETHEUS_URL=http://wg-monitoring:9090
    volumes:
      - ./grafana_data:/var/lib/grafana

networks:
  monitoring:
    driver: bridge

One important subtlety: because Prometheus uses network_mode: "service:wg-monitoring", it inherits the VPN container's network stack and cannot declare its own ports: block. To reach the Prometheus UI from your LAN, you must publish port 9090 on the wg-monitoring service — that's why the ports list there now has two entries instead of one.

How the Connection Works

When you log into Grafana at http://hub-ip:3000, you simply add a Prometheus data source pointing to http://wg-monitoring:9090.

Grafana sends a query to the wg-monitoring container over the monitoring bridge network.
Prometheus, which shares the same network stack, intercepts that request on port 9090.
Prometheus then looks at its config, sees the target 10.10.0.2, and routes the scrape request through the wg0 interface.
The encrypted traffic travels across the public internet, hits the remote VPN client, and pulls metrics from the exporters.

Zero metrics ports are exposed on the remote servers, and the Hub exposes nothing to the public internet beyond the VPN listener itself.

Wrapping Up Part 2

We now have a complete monitoring pipeline: remote agents collect metrics, Prometheus scrapes them through an encrypted tunnel, and Grafana renders dashboards without ever touching the public IPs of the remote nodes.

A note on access: the Grafana UI (3000) and Prometheus UI (9090) are bound to the Hub's host ports, but they are not meant to be reachable from the public internet. In a typical home-lab setup, you access them either from inside your LAN, or by tunneling into the LAN over a separate user-facing VPN. The only port the Hub actually offers to the outside world is the AmneziaWG listener on UDP 51820.

However, this setup still has a major weakness. If your server reboots, Docker might start Prometheus before the VPN interface is actually up, leading to a "blind" stack that needs manual restarts.

In Part 3, we will tackle these Race Conditions with Docker Healthchecks and wire up Alertmanager so failures actually reach you — over Telegram, email, or whatever channel you prefer. In Part 4, we will add Loki and Promtail to aggregate logs through the same encrypted tunnel, so metrics and logs finally live in one place.

Remote Server Monitoring over VPN: A Docker Approach (Part 1)

Alex Miller — Thu, 16 Apr 2026 17:29:11 +0000

The Dilemma of Remote Monitoring

If you've ever tried to set up a monitoring stack for a scattered infrastructure—say, a local home server, a cheap VPS in Europe, and a ZimaBoard running at your parents' house - you've probably faced the ultimate dilemma: How do I collect metrics securely?

The classic monitoring stack involves Prometheus pulling data from targets running agents like Node Exporter or cAdvisor. But leaving ports like 9100 or 8080 wide open to the public internet is a disaster waiting to happen. Scanners will find them within hours, leading to garbage traffic, leaked system information, or worse.

Many developers solve this by installing a VPN (like WireGuard) directly on the host OS. But this approach has major flaws: it mixes your personal VPN traffic with technical traffic, requires messy iptables routing, and breaks the clean isolation that Docker is supposed to provide.

In this series, I'll show you an elegant way to connect isolated monitoring components into a single, secure network using the VPN Sidecar pattern and Docker's Network Namespace sharing.

The Architecture Concept

Instead of configuring a VPN at the OS level, we will isolate WireGuard (or AmneziaWG, if you need to bypass DPI) inside a lightweight Docker container on our Hub server.

Here is the magic trick: We won't give our monitoring services (like Prometheus) their own Docker networks. Instead, we will force them to use the VPN container's network stack.

To Prometheus, it will look like it's natively sitting inside the VPN subnet (e.g., 10.10.0.x). It will be able to scrape remote nodes using their internal VPN IPs without routing a single byte through the public internet, and without exposing any host ports.

Step 1: Setting Up the VPN Hub

Let's start by creating the foundation on our local server (the Hub), which will store the metrics.

Create a directory for your project and add a docker-compose.yaml file:

services:
  wg-monitoring:
    image: amneziavpn/amneziawg-go:latest
    container_name: wg-monitoring
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv4.conf.all.src_valid_mark=1
    devices:
      - /dev/net/tun
    environment:
      - TZ=EST
    ports:
      - "51820:51820/udp" # Publish port to the host machine for incoming connections
    volumes:
      - ./config/wg0.conf:/etc/amnezia/amneziawg/wg0.conf
    restart: unless-stopped
    command: ["/bin/sh", "-c", "awg-quick up wg0 && sleep infinity"]
    networks:
      - monitoring

networks:
  monitoring:
    driver: bridge

What's happening here?

We grant NET_ADMIN capabilities so the container can create the wg0 network interface.
We expose UDP port 51820 so our remote servers can connect to this Hub.
We mount a volume containing our standard wg0.conf (your WireGuard/AmneziaWG server configuration).

Let's assume our Hub is assigned the IP 10.10.0.1 inside the VPN.

Step 2: The Network Namespace Magic

Now, let's add a service to the same docker-compose.yaml that actually needs to access the remote servers. Let's add Prometheus:

prometheus:
  image: prom/prometheus
  container_name: prometheus
  # WARNING: This is where the magic happens!
  network_mode: "service:wg-monitoring"
  depends_on:
    - wg-monitoring
  volumes:
    - ./prometheus:/etc/prometheus
  restart: unless-stopped

Notice the key line: network_mode: "service:wg-monitoring".

Because of this line, we do not define a networks or ports block for Prometheus. From a networking perspective, the prometheus container now shares the exact same network stack as wg-monitoring.

If you were to exec into the Prometheus container and run ip a, you would see the wg0 interface with the IP 10.10.0.1. Prometheus can now directly reach any client connected to this VPN.

Step 3: Securing the Remote Nodes

On your remote server (let's call it Node 1), we need to deploy a VPN client container alongside our metrics agents.

Here is the docker-compose.yaml for the remote server:

services:
  wg-client:
    image: amneziavpn/amneziawg-go:latest
    container_name: wg-client
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv4.conf.all.src_valid_mark=1
    devices:
      - /dev/net/tun
    environment:
      - TZ=EST
    volumes:
      - ./config/wg0.conf:/etc/amnezia/amneziawg/wg0.conf # Client config with IP 10.10.0.2
    restart: unless-stopped
    command: ["/bin/sh", "-c", "awg-quick up wg0 && sleep infinity"]

  node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    network_mode: "service:wg-client"
    depends_on:
      - wg-client
    restart: unless-stopped
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--path.rootfs=/rootfs'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'

We apply the exact same pattern here: node-exporter shares the network namespace with wg-client.

Now, Node Exporter is happily collecting host metrics and serving them on port 9100 — but only inside the VPN tunnel at 10.10.0.2. Not a single port scanner on the public internet can reach your metrics, because the port is never published to the host OS.

Wrapping Up Part 1

We have successfully created an invisible, encrypted L3 bridge between containers running on servers that might be thousands of miles apart. We achieved this cleanly without littering our host systems with complex iptables rules, maintaining the true "Infrastructure as Code" spirit.

In Part 2, we will dive into configuring Prometheus to scrape these remote targets through the tunnel, and we'll set up Grafana so you can safely view your dashboards via the web without exposing your database.