DEV Community

vast cow
vast cow

Posted on

Automatically Renewing Tailscale HTTPS Certificates on PiKVM

It is appropriate to continue using

ssl_certificate /etc/kvmd/nginx/ssl/server.crt;
ssl_certificate_key /etc/kvmd/nginx/ssl/server.key;
Enter fullscreen mode Exit fullscreen mode

in /etc/kvmd/nginx/ssl.conf, with a systemd timer checking the certificate expiration and updating these two files only when necessary.

The official PiKVM documentation also describes placing the Tailscale certificate in /etc/kvmd/nginx/ssl/server.{crt,key}, setting the group to kvmd-nginx, and then restarting kvmd-nginx. (Pikvm) Also, certificates obtained as files using tailscale cert are not automatically renewed, so users need to implement their own renewal process. --min-validity is also officially available in the current CLI. (Tailscale)

Configuration

Normally, the setup looks like this.

Tailscale
   │
   │ 100.x / MagicDNS
   ▼
PiKVM nginx :443
   │
   ├─ /etc/kvmd/nginx/ssl/server.crt
   └─ /etc/kvmd/nginx/ssl/server.key
Enter fullscreen mode Exit fullscreen mode

Do not use tailscale serve.

tailscale serve --https=443 off
Enter fullscreen mode Exit fullscreen mode

The certificate renewal process will be:

Timer runs once a day
        │
        ▼
Check current server.crt
        │
        ├─ FQDN is correct
        │  and at least 30 days remain
        │       → Do nothing
        │
        └─ Less than 30 days / no certificate / hostname mismatch
                │
                ▼
               rw
                │
                ▼
        tailscale cert
                │
                ▼
        Validate cert/key
                │
                ▼
        Replace nginx files
                │
                ▼
        nginx -t
                │
                ▼
        restart kvmd-nginx
                │
                ▼
               ro
Enter fullscreen mode Exit fullscreen mode

Let’s Encrypt certificates are valid for 90 days, so attempting renewal starting 30 days before expiration provides plenty of margin. (Tailscale)


1. Renewal Script

Create /usr/local/sbin/pikvm-tailscale-cert-renew.

#!/usr/bin/env bash
set -Eeuo pipefail

export PATH=/usr/local/sbin:/usr/local/bin:/usr/bin

CERT="/etc/kvmd/nginx/ssl/server.crt"
KEY="/etc/kvmd/nginx/ssl/server.key"

# 30 days
MIN_VALIDITY_SECONDS=$((30 * 24 * 60 * 60))
TS_MIN_VALIDITY="720h"

TMP=""
MADE_RW=0


log() {
    echo "pikvm-tailscale-cert-renew: $*"
}


cleanup() {
    rc=$?

    trap - EXIT INT TERM

    rm -f "${CERT}.new" "${KEY}.new" 2>/dev/null || true

    if [[ -n "${TMP:-}" ]]; then
        rm -rf "$TMP"
    fi

    if (( MADE_RW )); then
        sync

        if ! ro; then
            log "ERROR: failed to restore read-only filesystem"
            rc=1
        fi
    fi

    exit "$rc"
}

trap cleanup EXIT INT TERM


#
# Get the Tailscale FQDN
#
DOMAIN="$(
    tailscale status --json |
        jq -er '.Self.DNSName | rtrimstr(".") | select(length > 0)'
)"

log "Tailscale DNS name: ${DOMAIN}"


#
# Check the certificate currently used by nginx
#
cert_is_current() {
    [[ -s "$CERT" ]] || return 1
    [[ -s "$KEY" ]] || return 1

    # Check whether the hostname matches
    openssl x509 \
        -in "$CERT" \
        -noout \
        -checkhost "$DOMAIN" \
        >/dev/null 2>&1 || return 1

    # Check whether at least 30 days remain
    openssl x509 \
        -in "$CERT" \
        -noout \
        -checkend "$MIN_VALIDITY_SECONDS" \
        >/dev/null 2>&1 || return 1

    return 0
}


if cert_is_current; then
    log "certificate is valid for more than 30 days; nothing to do"
    exit 0
fi

log "certificate renewal is required"


#
# Use /tmp for the temporary directory.
# The root filesystem is still RO at this point.
#
TMP="$(mktemp -d /tmp/pikvm-tailscale-cert.XXXXXX)"


#
# Switch the PiKVM root filesystem to RW only when necessary.
#
ROOT_OPTS="$(findmnt -no OPTIONS /)"

case ",${ROOT_OPTS}," in
    *,rw,*)
        log "root filesystem is already read-write"
        ;;
    *)
        log "switching root filesystem to read-write"
        rw
        MADE_RW=1
        ;;
esac


#
# Obtain the certificate from Tailscale.
#
# --min-validity=720h requests a certificate
# that is valid for at least 30 days.
#
log "requesting certificate for ${DOMAIN}"

tailscale cert \
    --min-validity="$TS_MIN_VALIDITY" \
    --cert-file="$TMP/server.crt" \
    --key-file="$TMP/server.key" \
    "$DOMAIN"


#
# Validate the obtained certificate
#

# hostname
openssl x509 \
    -in "$TMP/server.crt" \
    -noout \
    -checkhost "$DOMAIN"

# expiration
openssl x509 \
    -in "$TMP/server.crt" \
    -noout \
    -checkend "$MIN_VALIDITY_SECONDS"

# Verify that the certificate and private key have the same public key
if ! cmp -s \
    <(
        openssl x509 \
            -in "$TMP/server.crt" \
            -pubkey \
            -noout |
        openssl pkey \
            -pubin \
            -outform DER 2>/dev/null
    ) \
    <(
        openssl pkey \
            -in "$TMP/server.key" \
            -pubout \
            -outform DER 2>/dev/null
    )
then
    log "ERROR: certificate and private key do not match"
    exit 1
fi


#
# Back up the current certificate
#
if [[ -e "$CERT" ]]; then
    cp -a "$CERT" "$TMP/old.crt"
fi

if [[ -e "$KEY" ]]; then
    cp -a "$KEY" "$TMP/old.key"
fi


rollback() {
    log "rolling back certificate"

    if [[ -e "$TMP/old.crt" ]]; then
        cp -a "$TMP/old.crt" "$CERT"
    else
        rm -f "$CERT"
    fi

    if [[ -e "$TMP/old.key" ]]; then
        cp -a "$TMP/old.key" "$KEY"
    else
        rm -f "$KEY"
    fi
}


#
# Prepare the files for nginx, then rename them.
#
# nginx itself continues holding the old certificate until it is
# reloaded/restarted, so even if the crt/key files briefly do not match
# between the two renames, this does not affect the running nginx process.
#
install \
    -o root \
    -g kvmd-nginx \
    -m 0644 \
    "$TMP/server.crt" \
    "${CERT}.new"

install \
    -o root \
    -g kvmd-nginx \
    -m 0640 \
    "$TMP/server.key" \
    "${KEY}.new"

mv -f "${KEY}.new" "$KEY"
mv -f "${CERT}.new" "$CERT"


#
# Validate using the actual nginx configuration generated by PiKVM
#
if ! nginx -t -c /run/kvmd/nginx.conf; then
    log "ERROR: nginx configuration test failed"
    rollback
    exit 1
fi


#
# Restart according to the official PiKVM documentation.
#
if ! systemctl restart kvmd-nginx; then
    log "ERROR: kvmd-nginx restart failed"

    rollback

    # Attempt recovery after restoring the old certificate
    nginx -t -c /run/kvmd/nginx.conf || true
    systemctl restart kvmd-nginx || true

    exit 1
fi


log "certificate successfully installed"

openssl x509 \
    -in "$CERT" \
    -noout \
    -subject \
    -issuer \
    -dates

exit 0
Enter fullscreen mode Exit fullscreen mode

With this method, the normal daily operation consists only of:

openssl x509 -checkhost ...
openssl x509 -checkend ...
Enter fullscreen mode Exit fullscreen mode

so the root filesystem remains RO.

It switches to rw only when fewer than 30 days remain.

Additionally, because tailscale cert --min-validity=720h is used, Tailscale is also instructed to “return a certificate that is valid for at least 30 days.” This flag is part of the current Tailscale CLI specification. (Tailscale)


2. systemd Service

/etc/systemd/system/pikvm-tailscale-cert-renew.service

[Unit]
Description=Renew Tailscale TLS certificate for PiKVM nginx
Wants=network-online.target
After=network-online.target tailscaled.service
Requires=tailscaled.service

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/pikvm-tailscale-cert-renew
TimeoutStartSec=5min
Enter fullscreen mode Exit fullscreen mode

There is no need to add kvmd-nginx.service to Requires=.

The reason is that even if kvmd-nginx has stopped because of a broken certificate, this unit should still be able to repair the certificate independently and then run systemctl restart kvmd-nginx.


3. systemd Timer

/etc/systemd/system/pikvm-tailscale-cert-renew.timer

[Unit]
Description=Periodic Tailscale TLS certificate check for PiKVM

[Timer]
OnBootSec=15min
OnUnitActiveSec=1d
RandomizedDelaySec=30min
AccuracySec=1min
Unit=pikvm-tailscale-cert-renew.service

[Install]
WantedBy=timers.target
Enter fullscreen mode Exit fullscreen mode

Persistent=true is intentionally omitted here.

Since this configuration starts renewing a 90-day certificate 30 days before expiration, missing a single check while the device is powered off is not a problem. The certificate will be checked roughly 15–45 minutes after boot, and then approximately once per day thereafter.


4. Installation

Switch PiKVM to RW only while creating the configuration files.

rw

chmod 755 /usr/local/sbin/pikvm-tailscale-cert-renew

systemctl daemon-reload

systemctl enable pikvm-tailscale-cert-renew.timer

ro
Enter fullscreen mode Exit fullscreen mode

Then completely disable tailscale serve.

tailscale serve --https=443 off
Enter fullscreen mode Exit fullscreen mode

PiKVM’s own nginx will listen on port 443.

The official PiKVM documentation also uses the approach of updating server.crt/server.key and running systemctl restart kvmd-nginx when installing a Tailscale certificate directly into nginx. (Pikvm)


5. Starting and Checking Status

systemctl start pikvm-tailscale-cert-renew.service
Enter fullscreen mode Exit fullscreen mode

Check:

systemctl status pikvm-tailscale-cert-renew.service
Enter fullscreen mode Exit fullscreen mode
journalctl \
    -u pikvm-tailscale-cert-renew.service \
    -n 100 \
    --no-pager
Enter fullscreen mode Exit fullscreen mode

Certificate:

openssl x509 \
    -in /etc/kvmd/nginx/ssl/server.crt \
    -noout \
    -subject \
    -issuer \
    -dates \
    -ext subjectAltName
Enter fullscreen mode Exit fullscreen mode

If it succeeds and contains:

DNS:{hostname}.{tsnet}.ts.net
Enter fullscreen mode Exit fullscreen mode

then everything is OK.

After that, start the timer.

systemctl start pikvm-tailscale-cert-renew.timer
Enter fullscreen mode Exit fullscreen mode

Check:

systemctl list-timers pikvm-tailscale-cert-renew.timer
Enter fullscreen mode Exit fullscreen mode

Access URL

With this configuration, the certificate name is:

{hostname}.{tsnet}.ts.net
Enter fullscreen mode Exit fullscreen mode

so in the browser, always use:

https://{hostname}.{tsnet}.ts.net/
Enter fullscreen mode Exit fullscreen mode

For https://{hostname}/ or [https://100.x.x.x/](https://100.x.x.x/), the connection itself may reach nginx, but the certificate name will not match. Tailscale also explicitly states that HTTPS certificates are for fully qualified *.ts.net names, not HTTPS certificates for bare hostnames. (Tailscale)

In other words, this approach completely eliminates Serve, handles port 443 using only PiKVM’s standard nginx, keeps the filesystem RO during normal operation, and switches it to RW only when the certificate actually needs to be renewed. It also does not conflict with the automatic generation of /run/kvmd/nginx.conf.

PiKVM で Tailscale HTTPS 証明書を自動更新する

/etc/kvmd/nginx/ssl.conf では、引き続き

ssl_certificate /etc/kvmd/nginx/ssl/server.crt;
ssl_certificate_key /etc/kvmd/nginx/ssl/server.key;
Enter fullscreen mode Exit fullscreen mode

を使用し、systemd タイマーで証明書の有効期限を確認して、必要な場合にのみこの 2 ファイルを更新する構成が適切です。

PiKVM の公式ドキュメントでも、Tailscale の証明書を /etc/kvmd/nginx/ssl/server.{crt,key} に配置し、グループを kvmd-nginx に設定したうえで kvmd-nginx を再起動する方法が説明されています。(Pikvm) また、tailscale cert でファイルとして取得した証明書は自動更新されないため、ユーザー側で独自の更新処理を実装する必要があります。現在の CLI では --min-validity も公式に利用できます。(Tailscale)

構成

通常、構成は次のようになります。

Tailscale
   │
   │ 100.x / MagicDNS
   ▼
PiKVM nginx :443
   │
   ├─ /etc/kvmd/nginx/ssl/server.crt
   └─ /etc/kvmd/nginx/ssl/server.key
Enter fullscreen mode Exit fullscreen mode

tailscale serve は使用しません。

tailscale serve --https=443 off
Enter fullscreen mode Exit fullscreen mode

証明書の更新処理は次のようになります。

タイマーを 1 日 1 回実行
        │
        ▼
現在の server.crt を確認
        │
        ├─ FQDN が正しく、
        │  かつ有効期限が 30 日以上残っている
        │       → 何もしない
        │
        └─ 残り 30 日未満 / 証明書なし / ホスト名不一致
                │
                ▼
               rw
                │
                ▼
        tailscale cert
                │
                ▼
        証明書/鍵を検証
                │
                ▼
        nginx のファイルを置換
                │
                ▼
        nginx -t
                │
                ▼
        kvmd-nginx を再起動
                │
                ▼
               ro
Enter fullscreen mode Exit fullscreen mode

Let’s Encrypt の証明書は 90 日間有効なので、有効期限の 30 日前から更新を試みれば十分な余裕があります。(Tailscale)


1. 更新スクリプト

/usr/local/sbin/pikvm-tailscale-cert-renew を作成します。

#!/usr/bin/env bash
set -Eeuo pipefail

export PATH=/usr/local/sbin:/usr/local/bin:/usr/bin

CERT="/etc/kvmd/nginx/ssl/server.crt"
KEY="/etc/kvmd/nginx/ssl/server.key"

# 30 日
MIN_VALIDITY_SECONDS=$((30 * 24 * 60 * 60))
TS_MIN_VALIDITY="720h"

TMP=""
MADE_RW=0


log() {
    echo "pikvm-tailscale-cert-renew: $*"
}


cleanup() {
    rc=$?

    trap - EXIT INT TERM

    rm -f "${CERT}.new" "${KEY}.new" 2>/dev/null || true

    if [[ -n "${TMP:-}" ]]; then
        rm -rf "$TMP"
    fi

    if (( MADE_RW )); then
        sync

        if ! ro; then
            log "ERROR: 読み取り専用ファイルシステムへの復元に失敗しました"
            rc=1
        fi
    fi

    exit "$rc"
}

trap cleanup EXIT INT TERM


#
# Tailscale の FQDN を取得
#
DOMAIN="$(
    tailscale status --json |
        jq -er '.Self.DNSName | rtrimstr(".") | select(length > 0)'
)"

log "Tailscale DNS 名: ${DOMAIN}"


#
# nginx が現在使用している証明書を確認
#
cert_is_current() {
    [[ -s "$CERT" ]] || return 1
    [[ -s "$KEY" ]] || return 1

    # ホスト名が一致しているか確認
    openssl x509 \
        -in "$CERT" \
        -noout \
        -checkhost "$DOMAIN" \
        >/dev/null 2>&1 || return 1

    # 有効期限が 30 日以上残っているか確認
    openssl x509 \
        -in "$CERT" \
        -noout \
        -checkend "$MIN_VALIDITY_SECONDS" \
        >/dev/null 2>&1 || return 1

    return 0
}


if cert_is_current; then
    log "証明書の有効期限は 30 日以上残っています。処理は不要です"
    exit 0
fi

log "証明書の更新が必要です"


#
# 一時ディレクトリには /tmp を使用。
# この時点ではルートファイルシステムはまだ RO。
#
TMP="$(mktemp -d /tmp/pikvm-tailscale-cert.XXXXXX)"


#
# 必要な場合にのみ PiKVM のルートファイルシステムを RW に切り替える。
#
ROOT_OPTS="$(findmnt -no OPTIONS /)"

case ",${ROOT_OPTS}," in
    *,rw,*)
        log "ルートファイルシステムはすでに読み書き可能です"
        ;;
    *)
        log "ルートファイルシステムを読み書き可能に切り替えます"
        rw
        MADE_RW=1
        ;;
esac


#
# Tailscale から証明書を取得。
#
# --min-validity=720h により、少なくとも 30 日間
# 有効な証明書を要求する。
#
log "${DOMAIN} の証明書を要求しています"

tailscale cert \
    --min-validity="$TS_MIN_VALIDITY" \
    --cert-file="$TMP/server.crt" \
    --key-file="$TMP/server.key" \
    "$DOMAIN"


#
# 取得した証明書を検証
#

# ホスト名
openssl x509 \
    -in "$TMP/server.crt" \
    -noout \
    -checkhost "$DOMAIN"

# 有効期限
openssl x509 \
    -in "$TMP/server.crt" \
    -noout \
    -checkend "$MIN_VALIDITY_SECONDS"

# 証明書と秘密鍵の公開鍵が同一であることを確認
if ! cmp -s \
    <(
        openssl x509 \
            -in "$TMP/server.crt" \
            -pubkey \
            -noout |
        openssl pkey \
            -pubin \
            -outform DER 2>/dev/null
    ) \
    <(
        openssl pkey \
            -in "$TMP/server.key" \
            -pubout \
            -outform DER 2>/dev/null
    )
then
    log "ERROR: 証明書と秘密鍵が一致しません"
    exit 1
fi


#
# 現在の証明書をバックアップ
#
if [[ -e "$CERT" ]]; then
    cp -a "$CERT" "$TMP/old.crt"
fi

if [[ -e "$KEY" ]]; then
    cp -a "$KEY" "$TMP/old.key"
fi


rollback() {
    log "証明書をロールバックしています"

    if [[ -e "$TMP/old.crt" ]]; then
        cp -a "$TMP/old.crt" "$CERT"
    else
        rm -f "$CERT"
    fi

    if [[ -e "$TMP/old.key" ]]; then
        cp -a "$TMP/old.key" "$KEY"
    else
        rm -f "$KEY"
    fi
}


#
# nginx 用のファイルを準備してからリネームする。
#
# nginx 自体はリロード/再起動されるまで古い証明書を保持し続けるため、
# 2 回のリネームの間に crt/key ファイルが一時的に一致しない状態になっても、
# 稼働中の nginx プロセスには影響しない。
#
install \
    -o root \
    -g kvmd-nginx \
    -m 0644 \
    "$TMP/server.crt" \
    "${CERT}.new"

install \
    -o root \
    -g kvmd-nginx \
    -m 0640 \
    "$TMP/server.key" \
    "${KEY}.new"

mv -f "${KEY}.new" "$KEY"
mv -f "${CERT}.new" "$CERT"


#
# PiKVM が生成した実際の nginx 設定を使用して検証
#
if ! nginx -t -c /run/kvmd/nginx.conf; then
    log "ERROR: nginx の設定テストに失敗しました"
    rollback
    exit 1
fi


#
# PiKVM 公式ドキュメントに従って再起動。
#
if ! systemctl restart kvmd-nginx; then
    log "ERROR: kvmd-nginx の再起動に失敗しました"

    rollback

    # 古い証明書を復元した後、復旧を試みる
    nginx -t -c /run/kvmd/nginx.conf || true
    systemctl restart kvmd-nginx || true

    exit 1
fi


log "証明書を正常にインストールしました"

openssl x509 \
    -in "$CERT" \
    -noout \
    -subject \
    -issuer \
    -dates

exit 0
Enter fullscreen mode Exit fullscreen mode

この方法では、通常の日次処理で実行されるのは次の確認だけです。

openssl x509 -checkhost ...
openssl x509 -checkend ...
Enter fullscreen mode Exit fullscreen mode

したがって、ルートファイルシステムは RO のまま維持されます

rw に切り替わるのは、有効期限の残りが 30 日未満になった場合だけです。

さらに、tailscale cert --min-validity=720h を使用しているため、Tailscale に対しても「少なくとも 30 日間有効な証明書を返す」よう指定しています。このフラグは現在の Tailscale CLI 仕様に含まれています。(Tailscale)


2. systemd サービス

/etc/systemd/system/pikvm-tailscale-cert-renew.service

[Unit]
Description=PiKVM nginx 用 Tailscale TLS 証明書を更新
Wants=network-online.target
After=network-online.target tailscaled.service
Requires=tailscaled.service

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/pikvm-tailscale-cert-renew
TimeoutStartSec=5min
Enter fullscreen mode Exit fullscreen mode

Requires=kvmd-nginx.service を追加する必要はありません。

理由は、証明書の破損によって kvmd-nginx が停止していたとしても、このユニットは独立して証明書を修復し、その後 systemctl restart kvmd-nginx を実行できるようにしておくべきだからです。


3. systemd タイマー

/etc/systemd/system/pikvm-tailscale-cert-renew.timer

[Unit]
Description=PiKVM 用 Tailscale TLS 証明書の定期チェック

[Timer]
OnBootSec=15min
OnUnitActiveSec=1d
RandomizedDelaySec=30min
AccuracySec=1min
Unit=pikvm-tailscale-cert-renew.service

[Install]
WantedBy=timers.target
Enter fullscreen mode Exit fullscreen mode

ここでは意図的に Persistent=true を省略しています。

この構成では、90 日間有効な証明書について有効期限の 30 日前から更新を開始するため、デバイスの電源が切れている間に 1 回チェックできなくても問題ありません。起動後およそ 15~45 分以内に証明書がチェックされ、その後はおよそ 1 日に 1 回チェックされます。


4. インストール

設定ファイルを作成している間だけ PiKVM を RW に切り替えます。

rw

chmod 755 /usr/local/sbin/pikvm-tailscale-cert-renew

systemctl daemon-reload

systemctl enable pikvm-tailscale-cert-renew.timer

ro
Enter fullscreen mode Exit fullscreen mode

その後、tailscale serve を完全に無効化します。

tailscale serve --https=443 off
Enter fullscreen mode Exit fullscreen mode

PiKVM 自身の nginx がポート 443 で待ち受けます。

PiKVM の公式ドキュメントでも、Tailscale の証明書を nginx に直接インストールする場合、server.crt/server.key を更新して systemctl restart kvmd-nginx を実行する方法が使用されています。(Pikvm)


5. 起動とステータス確認

systemctl start pikvm-tailscale-cert-renew.service
Enter fullscreen mode Exit fullscreen mode

確認:

systemctl status pikvm-tailscale-cert-renew.service
Enter fullscreen mode Exit fullscreen mode
journalctl \
    -u pikvm-tailscale-cert-renew.service \
    -n 100 \
    --no-pager
Enter fullscreen mode Exit fullscreen mode

証明書:

openssl x509 \
    -in /etc/kvmd/nginx/ssl/server.crt \
    -noout \
    -subject \
    -issuer \
    -dates \
    -ext subjectAltName
Enter fullscreen mode Exit fullscreen mode

成功し、次の内容が含まれていれば問題ありません。

DNS:{hostname}.{tsnet}.ts.net
Enter fullscreen mode Exit fullscreen mode

その後、タイマーを開始します。

systemctl start pikvm-tailscale-cert-renew.timer
Enter fullscreen mode Exit fullscreen mode

確認:

systemctl list-timers pikvm-tailscale-cert-renew.timer
Enter fullscreen mode Exit fullscreen mode

アクセス URL

この構成では、証明書名は次のようになります。

{hostname}.{tsnet}.ts.net
Enter fullscreen mode Exit fullscreen mode

したがって、ブラウザでは常に次を使用します。

https://{hostname}.{tsnet}.ts.net/
Enter fullscreen mode Exit fullscreen mode

https://{hostname}/ または https://100.x.x.x/ の場合、接続自体は nginx に到達する可能性がありますが、証明書名は一致しません。Tailscale も、HTTPS 証明書は完全修飾された *.ts.net 名を対象とするものであり、単純なホスト名に対する HTTPS 証明書ではないことを明示しています。(Tailscale)

つまり、この方法では Serve を完全に排除し、ポート 443 を PiKVM 標準の nginx のみで処理し、通常時はファイルシステムを RO のまま維持し、実際に証明書の更新が必要になった場合にのみ RW に切り替えることができます。また、/run/kvmd/nginx.conf の自動生成とも競合しません。

Top comments (0)