Sergej Voronko

Posted on Nov 11 Originally published at linuxcore.dev

Tailscale vs Headscale: Self-Hosting Control

#linux #tailscale #vpn #networking

Self-hosting Headscale is for homelab admins who require total sovereignty over their network control plane. For most users, Tailscale's official free tier offers enough capacity and removes the maintenance burden. This comparison breaks down the technical differences, deployment realities, and client configurations needed to run a fully self-hosted mesh network.

The Control Plane vs the Data Plane

Mesh VPNs replaced traditional hub-and-spoke networks by allowing devices to communicate directly. WireGuard made this practical by providing lightweight, high-performance encrypted tunnels. However, managing WireGuard keys across homelab servers, laptops, and mobile devices quickly becomes a configuration nightmare.

The proprietary Tailscale coordination server securely distributes public keys to all authorized nodes. Headscale implements the exact same API endpoints. When a client authenticates, Headscale transmits a network map to the client, including the public keys and IP addresses of other nodes. The client configures its local WireGuard interface using this data.

Headscale replaces only the coordination server. When you run Headscale, you still install the official Tailscale client software on your devices. You point those clients to your own server instead of the Tailscale cloud. The data plane remains exactly the same. Peer-to-peer tunnels never pass through your Headscale server unless they fall back to a relay.

Setting Up Headscale in Your Homelab

Headscale is distributed as a single Go binary or a Docker container. It requires a dedicated domain name and valid TLS certificates. Tailscale clients reject self-signed certificates without explicit overrides, making a valid Let's Encrypt certificate mandatory.

Here is a minimal Docker Compose stack for Headscale using a host-mounted directory for configuration and data:

services:
  headscale:
    image: headscale/headscale:latest
    container_name: headscale
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - ./config:/etc/headscale
      - ./data:/var/lib/headscale
    command: headscale serve

You must place a valid config.yaml in the configuration directory before starting the container. The default configuration uses SQLite for storage, which handles all Headscale use cases. PostgreSQL is supported for legacy reasons, but it is in maintenance mode and discouraged.

A reverse proxy like Caddy, Traefik, or Nginx should sit in front of Headscale to handle HTTPS termination on port 443 and proxy the traffic to port 8080. You must ensure the host server has a static IP address or reliable dynamic DNS. Because Headscale orchestrates the entire mesh network, its endpoint requires high availability.

Client Configuration and Device Registration

Pointing official clients to your custom server requires overriding the default login URL. On Linux, run the client with the login server flag:

tailscale up --login-server https://headscale.example.com

Mobile platforms require different steps. On iOS, open the Tailscale app, tap the options menu on the login screen, and select the custom coordination server option. Enter your Headscale URL. Instructions for connecting Apple devices are available at the /apple endpoint, with downloadable configuration profiles provided for macOS under /apple/{platform}.

Headscale provides no built-in web GUI for administration. You manage users and devices through the command line. To register a node, the client prints an authentication command. Run that command inside the Headscale container:

docker exec headscale headscale nodes register --user homelab --key nodekey:xxxxxxxxxxxxxxxxxxxx

Alternatively, generate a pre-authentication key and pass it to the client during setup:

docker exec headscale headscale preauthkeys create --user homelab --expiration 24h

Then supply this key to your client:

tailscale up --login-server https://headscale.example.com --authkey <your_key>

DERP Relaying and NAT Traversal

WireGuard connections often fail through restrictive firewalls or Carrier-Grade NAT. Tailscale implements Designated Encrypted Relay for Packets servers, commonly called DERP, to relay traffic when a direct peer-to-peer connection fails.

Tailscale provides a global network of DERP relays for free. Self-hosting Headscale means you lose access to this global network. Headscale includes an embedded DERP server, but it only exists in the single geographic location where you deployed the container. If you have nodes in Europe and Asia falling back to a relay in North America, latency will spike.

To support the embedded DERP server, you must expose UDP port 3478 and TCP port 443. For a true multi-region setup, you have to deploy and configure custom DERP nodes manually on separate virtual private servers and link them via the Headscale configuration file. This adds significant cost and maintenance overhead compared to the managed Tailscale service.

Access Control Lists and Identity

Tailscale uses a custom HuJSON format for Access Control Lists. Headscale parses this exact same format. Instead of editing policies in a web browser, you define your ACLs in a local file on the Headscale server and reference it in your main configuration. This approach works well for infrastructure-as-code workflows, allowing you to track network policies in a Git repository and deploy them alongside your server configuration.

Headscale also supports OpenID Connect. This allows you to integrate self-hosted identity providers like Authelia, Authentik, or Keycloak. Users authenticate against your own identity stack before joining the mesh network. This integration replaces the standard Google, Microsoft, or GitHub single sign-on options provided by the official Tailscale service. By pairing Headscale with a self-hosted OIDC provider, your entire authentication chain remains fully offline and under your direct control.

Maintenance vs Official Free Tier

Running Headscale means taking on the operational burden of a critical network service. If your Headscale server goes down, existing peer-to-peer tunnels remain active until keys expire, but no new devices can join and IP roaming breaks.

Maintaining Headscale involves tracking upstream Tailscale changes. Headscale developers frequently release updates to maintain compatibility with new Tailscale client versions. If you update your Tailscale clients on iOS or Android but leave Headscale running an outdated version, API mismatch errors occur and devices fail to authenticate.

You must implement automated backups for the data directory. If you lose the SQLite database and private keys, you have to rebuild the entire network from scratch and re-register every single device.

Tailscale offers a free Personal plan supporting up to 6 users and unlimited user devices. The official service provides high availability, global DERP relays, and a web-based admin console. Choose Headscale if your threat model forbids third-party metadata collection, or if you need to run an air-gapped network. Choose Tailscale for zero-maintenance operation. The decision comes down to the time you want to spend managing the network versus using the network.

What's Next

FAQ

Can Headscale use the official Tailscale client apps?

Yes. Headscale uses the exact same Tailscale clients for Linux, Windows, macOS, iOS, and Android. You point them to your custom coordination server instead of the default Tailscale infrastructure.

Does Headscale have a web interface?

Headscale operates entirely via the command line by default. Community projects like Headplane provide a web UI, but require deploying and connecting a separate Docker container.

Do I lose access to Tailscale MagicDNS with Headscale?

No. Headscale fully supports MagicDNS. You can configure your own base domain in the Headscale configuration file, and devices will resolve each other by hostname.

Originally published at linuxcore.dev.

DEV Community