ServeHub-2: A Self-Hosted Two-Server VPN Architecture

Roman Frolov — Tue, 07 Jul 2026 07:38:55 +0000

It all started when I became dissatisfied with my approach to deploying personal services. I decided to rewrite everything from scratch. Besides, it was an excellent opportunity to gain new skills and try out tools I hadn’t found the time for.

Renting powerful VPS instances for resource-heavy tasks is unjustifiably expensive in the current climate. Meanwhile, at home, I already had a dedicated nettops (mini-PC) with 16 GB of RAM powered by an energy-efficient Intel N95 processor.

Going from simple Bash scripts and third-party tunnels to a fully automated infrastructure, I created the ServeHub-2 project. In this article, I will detail how the project's network evolved, why the declarative approach triumphed over the imperative one, and what non-obvious bugs I encountered during the automation process.

Network Architecture Setup History

Using Tuna

At the very beginning of the project, I wasn't yet thinking about a VPN as a way to punch through NAT and serve as the foundation for the entire system. The first thing I arrived at was a service called Tuna. In short, it is a Cloudflare Tunnel analogue costing around 300 rubles. It has a very pleasant web interface and an incredibly simple setup. However, it wasn't suitable for a fully independent architecture for two reasons:

Tuna servers are outside the user's control.
You cannot deploy your own accompanying services on the remote server.

Overall, the service is indeed convenient, but it proved to be too limiting for my tasks.

In Search of Flexibility: Testing Pangolin and Moving to FRP

After Tuna, I decided to move towards having my own control and tried Pangolin. However, this solution was quickly dropped: it turned out to be too "heavy" and redundant for a cheap server. The main disadvantages were a noticeable resource overhead and an extra layer of forced web authentication in front of the services themselves. This broke normal interaction with native mobile clients (like Element for Matrix or Bitwarden for passwords), where repeated authorization in a browser is simply not needed.

It was replaced by FRP (Fast Reverse Proxy). It performed the same forwarding function, but I hosted it on my own rented VPS in Layer 4 (TCP) mode. This gave absolute flexibility: the external VPS did not look inside the packets or decrypt anything; it simply forwarded the raw stream home. Furthermore, this perfectly optimized expenses: instead of 300 rubles for Tuna and another 300 rubles for a separate VPN, I started paying just 500 rubles for a single stable VPS that I could configure however I wanted. (Moreover, it was possible to spend even less, since a VPS with 4 GB of RAM is only 40% loaded, and the CPU is at just 20%–40%).

Full Migration to WireGuard (AmneziaWG) and 8 Hours of Debugging

Over time, the architecture evolved into a full-fledged VPN network based on WireGuard—or more precisely, its modification, AmneziaWG. There were several reasons for this move:

Security: FRP still exposed internal resources to the public internet, leaving them vulnerable to port scanners.
Routing Convenience: All participants in the network became equal nodes within a single virtual local subnet. There was no longer a need to set up permanent one-way port forwards.
Additional Bonus: Since the remote VPS was purchased in the Netherlands, I automatically gained secure access to all foreign resources through this same VPN.

To implement this scheme, a wg-easy container with AmneziaWG support was deployed on the remote VPS, and an Amnezia client container (built from a Dockerfile using amnezia-tools and the host kernel module) was deployed on the home mini-PC.

And this is exactly where I caught the most exhausting bug of the project. After deployment, traffic stubbornly flowed in only one direction. It took about 8 hours of diagnosing iptables, routes, and reading foreign forums (which was useless, given the specific nature of our blocks). As it turned out, the provider was simply dropping the return traffic of standard WireGuard because the packets were sent without obfuscation. The reason lay in docker-compose.yml: I had specified image: ghcr.io/wg-easy/wg-easy:latest. As it unfolded, the latest tag on Docker Hub was stuck tightly to the old version 14.x, while AmneziaWG parameter support only appeared in the 15.x branch. Changing the tag to a specific version (15.3) solved the problem in a second.

Nginx Configuration

To ensure services are accessible exclusively within the VPN subnet, I utilized Nginx. Access to applications was tightly restricted at the configuration level—the web server only accepts requests from the 10.8.0.0/24 IP address range. Any attempts to knock on the server from the external internet without an active VPN tunnel are automatically dropped by Nginx.

The distribution logic relies on the proxy protocol: Nginx on the remote VPS acts as the main ingress node, receives the encrypted traffic, wraps it with headers containing the client's real IP address, and forwards it through the tunnel to the local Nginx on the home server. The local web server then decrypts the SSL itself and distributes the traffic to the destination Docker containers, keeping the real IPs in the security logs.

SSL Certificates

To obtain valid SSL certificates, I set up automated operation via Certbot using DNS-01 challenge validation with the Webnames API. The domain itself is pointed to the internal IP address 10.8.0.1. Verification via DNS allowed the issuance of a single wildcard certificate for the entire domain and its subdomains without needing to keep port 80 of the web server open to the outside.

The result is a setup where all services are accessible via beautiful domain names with HTTPS but are completely invisible to the external internet.

Software History

In parallel with the network structure, the application stack itself was also developing. Initially, I wanted to gather the tools I use every day in one place, but in the process of self-hosting, you quickly realize: you can't just throw containers together and hope the mini-PC handles it while the configuration files don't turn into a mess.

First Stack and Optimization

The first to settle on the home server were media services: Navidrome for music streaming and Audiobookshelf for audiobooks and podcasts. They are lightweight, have excellent mobile clients with progress synchronization, and fully cover my needs. Later, Nextcloud was added as a single independent cloud for files, contacts, and family documents.

Then came the question of secure password storage. At first, I looked at the original Bitwarden, but in the end, I chose Vaultwarden—an alternative server written in Rust that is fully compatible with the Bitwarden API. It consumes a mere few megabytes of RAM and works perfectly. Additionally, to easily manage this distributed Docker infrastructure, Portainer was added to the local stack (+ Portainer Agent on the remote server).

Design Mistakes: Why I Removed Matrix (Synapse)

Not all decisions stood the test of time. During the Tuna and FRP phases, I deployed a Matrix (Synapse) server for secure messaging. It seemed like a cool idea, but once I fully transitioned to AmneziaWG, the usefulness of a messenger inside a closed tunnel dropped to zero.

Synapse demanded too many resources, wasted the home PC's RAM, and complicated the Nginx config. At the same time, it brought no real benefit to the family. For critical infrastructure alerts and daily communication, it proved simpler and more efficient to use Telegram (plus, alerting is configured specifically through it). Ultimately, I completely purged Synapse from the stack, freeing up resources.

In its place, I added a local AdGuard Home to pair with wg-easy. Now it runs right inside the VPN network: it cleans all traffic from ads and trackers on the fly, caches DNS queries, and prevents web surfing history from leaking to external ISPs.

From Bash Crutches to Declarative Ansible

The entire stack on both servers was deployed using bash scripts, which was very bad from an idempotency perspective. First, because of bash, I constantly had to wipe the servers since I had no proper checks and didn't want to write them, and there were also many crutches involving importing variables, writing to files, and the like.

This brought me to a declarative approach and Ansible. Now the entire configuration is described as playbooks and roles reflecting the desired end state of the servers. Sensitive data has been moved to a secrets.yml file, and fragile structures are automated via Jinja2 templates. The project became idempotent: if steps are already completed, Ansible simply skips them.

This resulted in several yaml files for standard system configuration (bootstrap_os.yml) and for configuring each of the hosts (setup_local.yml and setup_remote.yml). Now, all you need to deploy the project completely from scratch is to download Ansible, a few other dependencies to your working PC, and run a single manage_deploy.sh, where you can choose how Ansible will behave and quietly wait for the services to deploy.

Also, thanks to Ansible, I conveniently implemented project portability. Since I decided not to use docker volumes and instead store everything in folders, to migrate old project data you just need to copy the apps-data folder, put it in the right place, and everything will work on its own after redeploying the project. A separate pause is allocated for this in Ansible.

Testing Multi-Distribution Compatibility with Vagrant

The project was initially designed to run on three distributions: Ubuntu, Debian, and Arch Linux. Testing Ansible playbooks directly on my working local machine (in my case, EndeavourOS) is too risky, and creating virtual machines by hand is slow and inconvenient.

The solution was Vagrant, allowing clean OS instances to be spun up in VirtualBox from pre-made images in a couple of minutes. However, during the configuration of the multi-vendor testbed in bridged mode (public_network), two critical issues surfaced:

DNS Conflict: By default, Vagrant creates a NAT interface to manage the node. When enabling the second (public) interface for the local network, the default DNS resolver broke. The issue had to be solved by forcing the cleanup and rewrite of the /etc/resolv.conf file via a Vagrant inline automation script.
GRUB Issue on Debian: In the base image used, generic/debian12, the GRUB configuration retained a hard link to the specific disk name from the builder's environment. Re-running playbooks on the testbed led to bootloader failures. To automate the cleanup, I had to implement a script that determines the system disk name on the fly via lsblk and automatically passes the correct parameters to the bootloader using the debconf-set-selections utility.

Reliable Backup System Based on Borgmatic

To create backups, I implemented Borgmatic (a convenient wrapper around the deduplicating tool Borg Backup). The entire process is automated using a combination of systemd units: borgmatic.service and borgmatic.timer. Two key directories go into the backup: apps-data (application configurations and databases) and PersonalData (media library: music, books, podcasts, Nextcloud files).

The deployment of the backup system is fully handled by Ansible. I only need to specify the UUID of the external hard drive—the script will check for its presence in the system, mount it to the correct directory, create an encrypted repository, and configure the rotation policy (keeping 7 daily, 4 weekly, and 6 monthly copies). Monitoring of the .borg repository itself is also exported to Prometheus to track its size and successful archiving status.

The main problem when backing up running Docker containers is the risk of copying a database in a "corrupted" or inconsistent state if active writing was happening at the moment the archive was created. To solve this problem, I utilized the hook mechanism in the Borgmatic configuration.

Before the backup begins, a command to stop the project containers (docker compose down) triggers automatically, and after successful completion (or in case an unexpected error occurs), the containers automatically spin back up in the background. For easy viewing of archives and fast file restoration, I deployed the Borg UI web interface.

Enterprise-Level Observability

In recent releases (v1.2.0 and v1.3.0), the project's focus shifted towards monitoring and log management.

Evolution of Alerting and Transition to Gatus

Initially, for availability monitoring, I looked at Uptime Kuma but rejected it due to the lack of convenient declarative configuration via files. Then I deployed a combination of Blackbox Exporter and Alertmanager with notifications sent to Matrix. But here lay a logical inconsistency: all alerting was tied to the local PC, and in the event of its hardware failure, I would simply lose all notifications.

Then I decided to bring back Uptime Kuma but deploy it on the remote VPS as an external "watchdog" and automate its configuration via a Python script using the uptime-kuma-api library. But "rakes" were waiting here too—the library hadn't been updated for three years and broke completely on fresh versions of Kuma.

Ultimately, the ideal solution was Gatus. It was originally designed to be managed via YAML configs and supports sending alerts if a server fails to respond. Due to the specifics of the Gatus frontend (it cannot work from a subdirectory like /gatus), I had to move it and wg-easy to full subdomains: gatus. and wireguard..

To monitor the hardware resources of both the remote and local servers, a combination of node-exporter and cAdvisor was deployed. All notifications now arrive instantly in a Telegram bot. To avoid a avalanche of identical messages (for example, during a host reboot), tight grouping and event deduplication are configured in Alertmanager. An exporter for AdGuard Home was also added, outputting blocked request statistics to Grafana.

Centralized Logs: Loki + Grafana Alloy

In release v1.3.0, the centralized log collection system Loki was added to the stack. Instead of the outdated Promtail, I applied Grafana Alloy as the collection agent.

It efficiently gathers logs from all running Docker containers, parses them, and forwards them to Loki. Now the entire history of events, Nginx web server errors, or internal application crashes is available in a single Grafana interface with convenient LogQL filtering capabilities, which significantly simplifies debugging.

Interface: Switching to Homepage

Initially, for the home page, I wrote a custom minimalist HTML panel with a Glassmorphism design. It looked beautiful, but adding new services manually by constantly editing the source code was extremely inconvenient.

Eventually, I replaced the self-written page with the full-fledged community project Homepage. This provided huge flexibility: the entire panel is configured via simple YAML files and supports built-in integration widgets. Right on the main screen, the status of Docker containers, free disk space, and activity graphs from AdGuard Home are now displayed. I visually divided the services into two clear groups: user services (Nextcloud, Navidrome) and administrative utilities (Grafana, Portainer, wg-easy). The widgets for remote services feature a neat note indicating that they run on a VPS and will remain accessible even if the local mini-PC turns off.

Conclusion

The ServeHub-2 project has come a long way to daily operation. The system allows a secure hybrid network to be deployed in minutes, while maintaining full control over personal data and security.

The entire codebase of the project, detailed documentation, and deployment instructions are open and available to the community:
🔗 GitHub Repository: https://github.com/canntstand/ServeHub-2

This is my first article and, at the same time, the first personal project to which I have dedicated so much time. I will be glad to see your feedback in the comments!

DEV Community: Roman Frolov