DEV Community

Cover image for gitleaks Cloudflare tunnel token leak: rotating it took my tunnel down
Christian Anderson
Christian Anderson

Posted on

gitleaks Cloudflare tunnel token leak: rotating it took my tunnel down

I keep a private GitHub repo that snapshots my homelab's live configuration every night: reverse proxy, cron jobs, systemd units, router settings, and the full docker inspect of every container on the NAS. The NAS has no compose files worth trusting, so docker inspect is the closest thing it has to configuration.

The repo was built to be safe to push, and it had several layers of protection. It still leaked a live Cloudflare tunnel token for nearly three weeks. Then, when I rotated that token, I took my own tunnel down. Both failures are worth writing up.

The scanner

On 24 September I added a weekly secret scan. It runs no model, only a script:

  • gitleaks v8.30.1, installed from the official release tarball and checked against the published checksums.txt, not piped from curl into a shell. In v8.30 the command is gitleaks git <repo>, not the older detect --source.
  • It lists every repo on my GitHub account (31, including archived and private ones), keeps bare mirrors of them, and scans full history with --redact.
  • The parser keeps only repo, file, line, short commit, rule and date. The secret value and the match are dropped, and the raw reports are deleted.
  • Findings are fingerprinted against a baseline, so later runs post only new ones plus a one-line "0 new" heartbeat.
  • A clone, fetch or scan failure is posted as "NOT scanned — unknown", never as clean.

To prove full-history scanning worked, I ran a positive control: a synthetic GitHub token committed and then deleted in a bare mirror. gitleaks found it.

The first real run scanned 31 of 31 repos with no errors and made one finding: rule generic-api-key, in the config repo, at line 7 of the NAS's cloudflared-connector container JSON. It was still present at HEAD. It was the real tunnel token.

How it got past the redactor

The backup script redacts values in two independent ways:

  1. By key name. Anything under a key matching pass, secret, token, api_key, auth, session, jwt, cookie, webhook, dsn and similar becomes <redacted:N>, where N is the length.
  2. By shape. Any opaque string of 24 or more token-ish characters is masked even under an innocent key, unless it contains a /. That exemption exists so filesystem paths survive.

On top of that, a leak guard aborts the commit if any known-live secret appears in the output.

None of those saw this token. Container environment variables were redacted one by one, by key. But the container's Cmd and Entrypoint were written into the JSON raw, with no redaction call at all. And I had put the token in Cmd myself. When the tunnel lost its config on 4 September, I recovered by starting a plain connector like this:

docker run -d --name cloudflared-connector --restart unless-stopped \
  cloudflare/cloudflared:latest tunnel --no-autoupdate run --token "$TOK"
Enter fullscreen mode Exit fullscreen mode

In docker inspect, that becomes an argv list: "tunnel", "--no-autoupdate", "run", "--token", "eyJ…". The token is its own list element, with no key attached. The key-name rule had nothing to match. The shape rule was never called on Cmd. The leak guard only knew the values from two specific env files, and the tunnel token was not in either. Every layer was looking somewhere else.

Fixing the redactor

The fix is a redact_argv() function, now applied to both Cmd and Entrypoint. It redacts three things:

  • the value that follows any flag whose name looks secret, which covers --token <value>;
  • the value part of --flag=value pairs, using the same key-name rules;
  • any argument of 32 or more token-shaped characters that does not start with /.

The last rule deliberately drops the / exemption. Base64 can contain /, so "has a slash, must be a path" is not safe for an argv item. The only exception left is an argument that starts with /, which really is a path. I recaptured and pushed, and line 7 now reads <redacted:184>.

That fixes HEAD, not history. The token was still in older commits, and it had also been printed in an AI agent's tool output while I investigated. The repo is private, but a private repo that holds a live credential is only as safe as every token that can read it. It needed rotating.

Rotating it, and the outage I caused

A Cloudflare tunnel token encodes the tunnel's secret. Rotating it means PATCHing the tunnel with a new random tunnel_secret, which kills the old token straight away, then fetching the new token with GET …/token. Then the connector on the NAS has to be recreated with the new token.

I also wanted the new token out of Cmd for good. cloudflared reads the TUNNEL_TOKEN environment variable, and my backup already redacts environment values whose key contains TOKEN. So the plan was to write an env file, recreate the container with --env-file, and stop passing any token on the command line.

My first two attempts failed at writing the env file. The NAS's app-data directory is root-only, and the SSH user's home directory is not writable either. The second attempt had already run docker rm -f on the running connector before it tried to write the file. So the old container was gone, the new one could not be started, and every public hostname behind the tunnel was down for about a minute.

What worked:

  • create the env file with mktemp under /tmp, with umask 077;
  • docker run --env-file <that file> … tunnel --no-autoupdate run, with no --token argument;
  • delete the file straight afterwards. Docker reads an env file once, at container creation, so it does not need to stay on disk.

The container's Cmd now holds no token. The connector came back healthy with 4 connections, and all five public hostnames returned 200 when tested through the Cloudflare edge rather than my LAN. My local DNS resolves those names to internal addresses, and a local test that never touches Cloudflare proves nothing.

The rotation script now follows that order, so the next rotation is a single command.

The rule I should have followed

Write the new credential before removing the thing that uses the old one. docker rm -f goes last.

The order I used put the destructive step before the step most likely to fail. Writing a file to a box you don't fully control is exactly that kind of step. If the write had come first, the failure would have cost nothing. The old container would still have been running, and still using the old token.

This case has a wrinkle. Once the tunnel secret is PATCHed, the old token is dead for new connections, so there is no fully safe window. That makes it more important, not less, that everything which can fail runs first: fetch the new token, write the env file, check it exists and is readable, and only then remove and recreate.

What I took from it

  • A redactor is only as good as the fields it is called on. Mine had two good rules and a leak guard, and none of them were applied to Cmd. When you add a new source of captured config, list every field and decide for each one whether it is redacted.
  • Credentials in argv are a trap in general. They show up in docker inspect, in ps, and in anything that snapshots a container's configuration. An env file doesn't make the secret invisible to someone with Docker access on the host, but it moves it to a field my tooling already treats as sensitive.
  • "Contains a slash, so it's a path" is not safe for secrets. It was a reasonable exemption for key-value config text. It was wrong for bare arguments.
  • Scan history, not just HEAD, and include a positive control. Without the planted-and-deleted test token, a clean first run would have told me nothing about whether history scanning worked.
  • Fixing HEAD is not remediation. If a secret was ever committed, rotate it.

The scan runs every Sunday morning. The next run should post "0 new". If it posts "NOT scanned", that means it couldn't check, not that everything is clean.


🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.

Top comments (0)