DEV Community

SEN LLC
SEN LLC

Posted on

I Built a .env Diff Tool That Refuses to Print Values

I Built a .env Diff Tool That Refuses to Print Values

A zero-dependency TypeScript CLI that diffs two .env files and tells you which keys were added, removed, or changed — without ever printing the values. Because .env files are secrets by default, and a diff tool for them has to be default-deny.

šŸ“¦ GitHub: https://github.com/sen-ltd/env-diff

Screenshot

The problem nobody admits to

It's 3 AM. Staging is fine. Prod is on fire. You know — you know — the difference is something in the env. So you SSH in and do the thing every engineer has done a hundred times:

diff .env.staging .env.prod
Enter fullscreen mode Exit fullscreen mode

And then you freeze. Because the output of diff is going to contain every value of every key in both files. Your terminal scrollback has it now. If you piped it anywhere — a log aggregator, a chat message, a PR comment, a tee to a file — the secrets are sitting there. At best it's awkward. At worst it's a compliance incident, and "I was just debugging" is not a defense that survives a postmortem.

So you close your eyes and do the dance: diff .env.staging .env.prod | awk '{print $1}' to try to hide the values, except diff output shape means that doesn't work, so you end up writing a one-liner with comm and sort and cut -d= -f1, and then you realize you haven't actually compared the values at all — you've only checked which keys are present in which file, and the bug is that LOG_LEVEL=info on staging is LOG_LEVEL=debug on prod and your one-liner would have missed it entirely.

I wrote env-diff to fix this specific workflow. It's ~400 lines of TypeScript with zero runtime dependencies, ships as a 136 MB Docker image, and has exactly one opinion: values never get printed. Not in text output, not in JSON output, not when you pipe somewhere, not when you pass --format=github. The only way to see a raw value is to pass --show-values, which emits a loud warning to stderr before the first byte of output, so if you leave it on by accident in CI, it screams on every run.

Here's what the output looks like:

$ env-diff .env.staging .env.prod
env-diff .env.staging .env.prod

~ DATABASE_URL  (8a3f12c7 -> 4e91b2d5)
~ LOG_LEVEL     (06271baf -> 4d1e008c)
+ SENTRY_DSN    (c8a29e4f)

summary: +1 added, -0 removed, ~2 changed, =3 same, 0 excluded
Enter fullscreen mode Exit fullscreen mode

No values. But you can see exactly which keys drifted, and you can see that DATABASE_URL is definitely different on the two sides — the short fingerprints don't match. That's the whole idea.

The hash-as-proof-of-difference trick

The core observation: to tell someone "these two secrets are different" without revealing either secret, you don't need the actual values — you need witnesses that are different when the values are different and the same when they're the same.

A hash does that. SHA-256 over UTF-8 bytes of each value:

// src/hasher.ts
import { createHash } from 'node:crypto';

export const HASH_LENGTH = 8;

export function hashValue(value: string): string {
  return createHash('sha256')
    .update(value, 'utf8')
    .digest('hex')
    .slice(0, HASH_LENGTH);
}
Enter fullscreen mode Exit fullscreen mode

I truncate to 8 hex chars (32 bits). Two reasons:

  1. 64 characters of hex is noise. In a terminal you want to compare fingerprints at a glance, not scan for the first byte where two blobs disagree.
  2. 32 bits is plenty for the job. For a file with under 100 keys, the birthday-collision probability per pair is roughly n² / 2^33, which is "worry about cosmic rays first" territory. If you're operating on a scale where that bothers you, you're also not using a command-line diff tool.

I want to be very clear about what this trick is and is not:

  • It is a way to prove that two values differ without printing them. If hashValue(a) !== hashValue(b), the values are different (collision aside).
  • It is not a way to keep the value secret from someone who sees the hash. For a low-entropy value like true, 8080, or info, 8 hex chars of SHA-256 can be rainbow-tabled in half a second. The tool isn't claiming preimage resistance. The tool is claiming "the raw value never leaves this process's memory unless you explicitly ask for it." Those are different promises.

The safety property is "values are never printed." The hash is just a witness that lets the diff stay useful even when the values are gone.

Why --show-values has to be loud

I thought about this flag for a long time. The obvious design is:

env-diff a.env b.env            # safe by default
env-diff a.env b.env -v         # show values
Enter fullscreen mode Exit fullscreen mode

That's the wrong design. -v on every other CLI means "verbose", and "verbose" normally means "print more stuff, no big deal". You'd get people aliasing env-diff to env-diff -v in their shell config because they find the hashes cryptic, and then six months later their CI starts running that alias through a Jenkins wrapper, and now half the environment is in build logs.

So the flag is:

  1. Long only. --show-values, no -v short form.
  2. Spelled out. You can't --show or --values — it's the full phrase, which means anyone reading your shell history sees exactly what you asked for.
  3. Loud. Before writing any output, the tool prints this to stderr:
   WARNING: --show-values is enabled. Actual .env values will be printed.
   Make sure the output is not being captured to a place secrets should not live.
Enter fullscreen mode Exit fullscreen mode

Not colored. Not silenceable. Always stderr so it shows up in CI logs even if stdout is captured.

  1. Never honored in --format github. GitHub Actions annotations go to PR pages, workflow summaries, and third-party observability tools. Nothing on that path is an appropriate home for a secret. So --format github --show-values silently drops the values from annotations. Is that surprising? Yes. Is it safer than the alternative? Also yes. The README documents it.

This is what security people mean by deliberate friction. The flag exists because it's occasionally necessary. It's annoying to use because it's occasionally necessary in ways that would be catastrophic if they became routine. Easy things should be safe; unsafe things should be hard.

The .env parser (don't use dotenv)

I deliberately did not pull in dotenv for parsing. A single dependency that does something this small is a forever-maintenance liability. But I also had to handle the edge cases that dotenv handles, because real .env files have them.

The parser is about 120 lines in src/parser.ts. Here's the core loop:

export function parseEnv(text: string): ParseResult {
  const entries = new Map<string, string>();
  const malformedLines: number[] = [];

  const lines = text.replace(/\r\n?/g, '\n').split('\n');

  for (let i = 0; i < lines.length; i++) {
    const line = stripLeading(lines[i]);
    if (line === '' || line.startsWith('#')) continue;

    const afterExport = stripExportPrefix(line);
    const eq = afterExport.indexOf('=');
    if (eq <= 0) {
      malformedLines.push(i + 1);
      continue;
    }

    const key = afterExport.slice(0, eq).trim();
    if (!isValidKey(key)) {
      malformedLines.push(i + 1);
      continue;
    }

    entries.set(key, parseValue(afterExport.slice(eq + 1)));
  }

  return { entries, malformedLines };
}
Enter fullscreen mode Exit fullscreen mode

Things the parser handles that I didn't expect to need:

  • export KEY=value. People copy lines from shell scripts into .env files and the export prefix sneaks in. dotenv handles it silently; so do I.
  • Trailing # comments on unquoted values. PORT=8080 # internal API should parse as PORT=8080, not PORT=8080 # internal API. But HASHY=abc#def (no space before the #) should parse as abc#def, because that's a legal URL fragment. The disambiguation is "the # has to be preceded by whitespace". Took me two tries.
  • Double-quoted strings with escape sequences. MSG="line1\nline2" should give you a literal newline inside the value. I roll my own tiny state machine for this — JSON.parse('"' + s + '"') feels clever but would mis-handle unescaped backslashes, and I don't want to ship a bug because I was saving lines.
  • Single-quoted strings. These are literal: RAW='$PATH' gives you the string $PATH, not a shell expansion. Docker Compose behaves the same way.
  • CRLF line endings. Windows developers exist. The parser normalizes \r\n and bare \r to \n before splitting.
  • Values with = in them. TOKEN=a=b=c should give a=b=c — the first = is the separator, everything else is part of the value.

Things the parser deliberately does not handle:

  • Variable interpolation. FOO=$BAR gives you the literal string $BAR. Interpolation behavior differs between dotenv, docker-compose, and systemd; picking one would silently change the semantics for users of the others. For a diff tool, I'd rather show "FOO has literally these bytes on both sides" than "FOO resolves to the same thing after running two different substitution engines".
  • Multi-line values. No \-continuation, no heredocs. .env files that need multi-line values usually need base64 anyway.

Malformed lines are reported with their 1-indexed line number, and the parser keeps going. A single garbage line in line 27 of a 400-line file doesn't make the whole file unparseable.

The differ

Once both files are parsed into Map<string, string>, the diff is easy:

// src/differ.ts (excerpt)
export function diffEnv(
  a: Map<string, string>,
  b: Map<string, string>,
  opts: DiffOptions = {},
): DiffResult {
  const excludeSet = new Set(opts.exclude ?? []);
  const excludePattern = opts.excludePattern;
  const isExcluded = (key: string) =>
    excludeSet.has(key) ||
    (excludePattern ? excludePattern.test(key) : false);

  const allKeys = new Set<string>();
  for (const k of a.keys()) allKeys.add(k);
  for (const k of b.keys()) allKeys.add(k);

  const entries: DiffEntry[] = [];
  const counts = { added: 0, removed: 0, changed: 0, same: 0, excluded: 0 };

  for (const key of [...allKeys].sort()) {
    if (isExcluded(key)) { counts.excluded++; continue; }

    const inA = a.has(key);
    const inB = b.has(key);

    if (inA && !inB) {
      const valueA = a.get(key)!;
      entries.push({ kind: 'removed', key, hashA: hashValue(valueA), valueA });
      counts.removed++;
    } else if (!inA && inB) {
      const valueB = b.get(key)!;
      entries.push({ kind: 'added', key, hashB: hashValue(valueB), valueB });
      counts.added++;
    } else if (inA && inB) {
      const valueA = a.get(key)!, valueB = b.get(key)!;
      const hashA = hashValue(valueA), hashB = hashValue(valueB);
      if (hashA === hashB) {
        entries.push({ kind: 'same', key, hash: hashA });
        counts.same++;
      } else {
        entries.push({ kind: 'changed', key, hashA, hashB, valueA, valueB });
        counts.changed++;
      }
    }
  }

  return { entries, counts };
}
Enter fullscreen mode Exit fullscreen mode

Two things worth noting:

First, even though the differ carries valueA and valueB on every changed entry, the formatters never print them unless showValues: true is passed. I considered stripping the values at diff time so they can't leak, but that would make --show-values impossible. Instead, the values live inside the structure, and each formatter is responsible for not emitting them by default. That puts the safety check at the single narrow boundary where output actually happens, which is easier to audit than scattering the check across call sites.

Second, --exclude and --exclude-pattern are both supported because CI pipelines inject keys that are supposed to differ between environments — BUILD_ID, CI_JOB_ID, DEPLOYED_AT_TIMESTAMP. Without a way to ignore them, every run would be a noisy "yes, these changed, because they always change", and people would learn to ignore the tool. The README suggests a .env-diffignore convention, but for now it's just flags.

GitHub Actions annotations

The --format github output emits GitHub Actions workflow commands:

// src/formatters/github.ts (excerpt)
export function formatGithub(
  result: DiffResult,
  opts: GithubFormatOptions,
): string {
  const lines: string[] = [];

  for (const entry of result.entries) {
    switch (entry.kind) {
      case 'added':
        lines.push(
          `::warning title=env-diff::key ${escape(entry.key)} added in ` +
            `${escape(opts.labelB)} (hash ${entry.hashB})`,
        );
        break;
      case 'removed':
        lines.push(
          `::warning title=env-diff::key ${escape(entry.key)} removed from ` +
            `${escape(opts.labelB)} (was in ${escape(opts.labelA)}, ` +
            `hash ${entry.hashA})`,
        );
        break;
      case 'changed':
        lines.push(
          `::warning title=env-diff::key ${escape(entry.key)} ` +
            `changed ${entry.hashA} -> ${entry.hashB}`,
        );
        break;
    }
  }

  lines.push(
    `::notice title=env-diff summary::+${result.counts.added} added, ` +
      `-${result.counts.removed} removed, ~${result.counts.changed} changed`,
  );

  return lines.join('\n') + '\n';
}

function escape(s: string): string {
  return s.replace(/%/g, '%25').replace(/\r/g, '%0D').replace(/\n/g, '%0A');
}
Enter fullscreen mode Exit fullscreen mode

A workflow command is any line in the form ::command parameter=value::message printed to stdout. GitHub's runner reads them while the job runs and turns ::warning:: into a yellow annotation on the PR "Files changed" view. A ::notice:: is blue and meant for informational summaries.

The escape function handles a subtlety in the workflow command protocol: %, \r, and \n in the message are interpreted as encoding markers, so they have to be percent-escaped. Without that, a key name containing a literal % (unusual but legal in some parsers) would produce broken annotations — or worse, allow message injection.

The point is that annotations are visible where humans actually look when reviewing a PR, not buried in a log file nobody reads. You can wire env-diff --format github --fail-on-diff into a CI step that compares .env.example against a CI-controlled .env.ci, and the first time someone adds a new env key without updating the example file, the PR lights up yellow before a reviewer has to remember to check.

Tradeoffs and things I won't add

  • Truncation collision. Eight hex chars is 32 bits. For a pair of values with the same hash, you have about a 1 in 4 billion chance of a false "same" per pair. For a file with 50 keys, that's still vanishingly small, but it's not zero. If you need a stronger guarantee, pass --format json and compare the entries programmatically — the full hashes are computable from the same code, I just don't surface them because 16 hex chars is where most of the marginal value has gone and the next 48 are just line noise.
  • No variable interpolation. Mentioned above. I'm not picking sides between dotenv, compose, systemd, and direnv.
  • No multi-line values. A .env file that needs a 10-line PEM block probably shouldn't be a .env file.
  • No colorized output. I considered it. But this tool's most important job is to be safe to pipe somewhere, and ANSI escape codes in a log pipeline are a papercut I'd rather not introduce. Plus, the text format is already terse enough to read without color.
  • No diff-against-schema mode. Some people will want env-diff --schema .env.schema .env.prod to check that every required key is present. That's a different tool — envcheck from the same portfolio does exactly that, and the two can compose.

Try it in 30 seconds

mkdir /tmp/edtest && cd /tmp/edtest
cat > a.env << 'EOT'
DATABASE_URL=postgres://localhost/a
API_KEY=alpha_key_12345
LOG_LEVEL=info
EOT
cat > b.env << 'EOT'
DATABASE_URL=postgres://localhost/b
API_KEY=alpha_key_12345
LOG_LEVEL=debug
NEW_FEATURE_FLAG=true
EOT

docker run --rm -v "$PWD":/work ghcr.io/sen-ltd/env-diff /work/a.env /work/b.env
# ~ DATABASE_URL  (2f99fe02 -> 146593f6)
# ~ LOG_LEVEL     (06271baf -> 0b8e9e99)
# + NEW_FEATURE_FLAG  (b5bea41b)
#
# summary: +1 added, -0 removed, ~2 changed, =1 same, 0 excluded
Enter fullscreen mode Exit fullscreen mode

No network calls, no telemetry, no latest tag that could swap under you — pin to a digest if you care. The whole thing is MIT-licensed, 47 vitest tests, and I'd be happy to review a PR that adds a sensible .env-diffignore format.

Closing thought

The reason I keep writing small security-shaped CLIs like this is that the defaults are the entire product. Nobody reads a flag's help text at 3 AM. Nobody remembers whether -v is "verbose" or "values" when prod is down. The whole point of a tool called env-diff is that a bleary-eyed engineer types env-diff .env.staging .env.prod, the right thing happens, and no secret ends up anywhere it shouldn't. Everything else — the JSON formatter, the GitHub annotations, the exclude patterns — is orbit around that one fact.

If this scratched an itch, the repo is at https://github.com/sen-ltd/env-diff. If it didn't, tell me why — my guess is either "I use direnv interpolation heavily" or "I actually do want the full hash" and both of those are solvable.

Top comments (0)