There is a file on this blog's server called /etc/mustafaerbay.env: API keys,
SMTP settings, a handful of numeric options. While reading through the deploy/
folder, something caught my eye — two different programs read that same file.
$ grep -rn '^EnvironmentFile=' deploy/
deploy/pipeline-health.service:8:EnvironmentFile=-/etc/mustafaerbay.env
deploy/mustafaerbay-digest.service:11:EnvironmentFile=-/etc/mustafaerbay.env
deploy/mustafaerbay-backup.service:10:EnvironmentFile=-/etc/mustafaerbay.env
deploy/mustafaerbay.service:21:EnvironmentFile=-/etc/mustafaerbay.env
$ grep -A1 'env_file' deploy/docker-compose.serve.yml
env_file:
- /etc/mustafaerbay.env
Four systemd units read it through EnvironmentFile=, the blog container through
Compose's env_file: key. Even the - in front of the path is a decision about
silence: per systemd's docs, if the file is missing it is not read and no error or
warning is logged. One file, two parsers. When I wrote this, I assumed the
two saw the same thing, because the file was called .env and I thought .env
was a format.
It isn't. .env is not a format, it is a habit. Every tool wrote its own rules
and those rules disagree. The gap turned out to be wide. A twenty-four-line file went to five different
parsers, and all five produced the same value on only four of those lines.
One file, five readers
They all read the same file with the same intent, and all their output lands in
the same place:
What the five share is the KEY=value line and a # at the start of a line.
Looking for anything beyond that — as I did — comes up empty.
The experiment: 24 lines, five parsers
Each line probes exactly one rule: quoting, comments, export, expansion,
escapes, whitespace, multi-line values. I put the lines in separate files, because
in a combined file a single bad line takes the rest down with it in some tools. I
also built one combined file: the first fourteen probes in a single file,
fourteen keys. That one shows up in the export section below.
The read commands, in order:
node --env-file=X.env -e 'console.log(JSON.stringify(process.env))'
/bin/sh -c 'set -a; . ./X.env; env'
docker run --rm --env-file X.env alpine:3.21 env
docker compose -f c.yml run --rm p # service: env_file: [X.env], command: env
systemd-run --user --wait --pipe --property=EnvironmentFile=/tmp/X.env /usr/bin/env
Versions: Node 22.16.0 (macOS, nvm), Docker 27.4.0 (build bde2b89), Compose
v2.31.0-desktop.2, alpine:3.21; the systemd side on my own server, 255
(255.4-1ubuntu8.18). I also re-ran the Node probes in containers on three more
versions — 20.20.2, 22.23.4, 24.22.2 — and all three produced identical output.
So the Node behaviour in the table below is not a version accident.
One caveat: the sh here is macOS's /bin/sh, that is bash 3.2.57 in POSIX mode.
That column does not measure a parser; it measures the shell executing the
file, and a different /bin/sh (dash, say) may behave differently on a few lines.
The other four columns are real parsers.
The table
Notation: (empty) means the value is an empty string, unset means the key was
never defined, REJECTED means the tool never read the file at all. ⏎ is a real
newline in the file, LF a newline inside the value, TAB a real tab character,
␠ a single space, and HOST the value inherited from the host environment.
| # | line in the file | node | sh | docker run | compose | systemd | same? |
|---|---|---|---|---|---|---|---|
| 1 | PLAIN=merbay |
merbay | merbay | merbay | merbay | merbay | yes |
| 2 | DQUOTE="merbay" |
merbay | merbay | "merbay" |
merbay | merbay | – |
| 3 | SQUOTE='merbay' |
merbay | merbay | 'merbay' |
merbay | merbay | – |
| 4 | BACKTICK=`merbay` | merbay | (empty) | `merbay` | `merbay` | `merbay` | – |
| 5 | export EXPORTED=merbay |
merbay | merbay | REJECTED | merbay | unset | – |
| 6 | INLINE=merbay # yorum |
merbay | merbay | merbay # yorum |
merbay | merbay # yorum |
– |
| 7 | TIGHT=merbay#yorum |
merbay | merbay#yorum |
merbay#yorum |
merbay#yorum |
merbay#yorum |
– |
| 8 | HASHQ="a # b" |
a # b |
a # b |
"a # b" |
a # b |
a # b |
– |
| 9 | SQHASH='a # b' |
a # b |
a # b |
'a # b' |
a # b |
a # b |
– |
| 10 | COLON: merbay |
unset | unset | REJECTED | merbay | unset | – |
| 11 | SPACED = merbay |
merbay | unset | REJECTED | merbay | merbay | – |
| 12 | SPACEQ= "merbay" |
merbay | unset | ␠"merbay" |
merbay | merbay | – |
| 13 | TRAIL=merbay␠␠␠ |
merbay | merbay | merbay␠␠␠ |
merbay | merbay | – |
| 14 |
BASE=merbay ⏎ INTERP=$BASE
|
$BASE |
merbay | $BASE |
merbay | $BASE |
– |
| 15 | CMDSUB=$(id -u) |
$(id -u) |
501 | $(id -u) |
$(id -u) |
$(id -u) |
– |
| 16 | RCE=`id -u` | id -u |
501 | `id -u` | `id -u` | `id -u` | – |
| 17 |
MULTI="bir ⏎ iki"
|
birLFiki |
birLFiki |
"bir |
birLFiki |
birLFiki |
– |
| 18 | ESCAPE="a\tb" |
a\tb |
a\tb |
"a\tb" |
aTABb |
a\tb |
– |
| 19 |
CONT=bir\ ⏎ iki
|
bir\ |
biriki |
bir\ |
bir\ |
biriki |
– |
| 20 | EMPTY= |
(empty) | (empty) | (empty) | (empty) | (empty) | yes |
| 21 | BAREVAR |
HOST | HOST | HOST | HOST | unset | – |
| 22 |
DUP=bir ⏎ DUP=iki
|
iki | iki | iki | iki | iki | yes |
| 23 |
; yorum ⏎ SEMI=merbay
|
merbay | unset | REJECTED | REJECTED | merbay | – |
| 24 | EQ=a=b |
a=b |
a=b |
a=b |
a=b |
a=b |
yes |
They agree on four lines: a plain assignment, an empty value, a duplicate key, and
an = inside the value. Of the remaining twenty lines, twelve have two distinct
readings and eight have three. Lines where one tool stands alone while the
other four agree: docker run 6, Node 1 (line 7), the shell 1 (line 15), systemd 1
(line 21). Compose never appears in that count, yet on two lines (10 and 18) it is
the one producing the only unique value of the five — on those lines the remaining
four disagree among themselves too, so it does not count as standing alone.
Now let's see where those differences come from. None of them is random — nearly
all of it is documented, just across five separate documents.
Quoting: is it "465" or 465?
Node, Compose and systemd strip a quote at the start and end of the value. Node's
docs say it accepts three quote characters: single, double and backtick. Compose's
docs show the same thing as VAR="VAL" -> VAL. systemd's docs devote a separate
paragraph each to single- and double-quoted values.
docker run --env-file never mentions quotes — and that is not an omission, it is
the decision itself: the documented file syntax is only <variable>=value,
<variable>, and a # at the start of a line. So if you write
DQUOTE="merbay", the value inside the container is "merbay", quotes included.
This is a quiet class of bug. When quotes stick to a connection string, the error
message usually says "host not found"; nobody shows you the quotes.
# is a comment in three different places
There are exactly three rules, and all three are written down:
-
Node: everything after a
#is a comment, with no space required before it. That is why line 7, which readsmerbay#yorumin the file, yieldsmerbay, and why Node stands alone there. -
Compose: for an unquoted value, an inline comment must be preceded by a
space. The docs spell this out with
VAR=VAL# not a comment. -
docker run and systemd: only a
#at the start of a line is a comment; a#in the middle is part of the value. Docker's docs say so in one sentence; systemd's say "lines starting with;or#".
What Node does when a quote is involved is not in the docs, so it had to be
measured: on lines 8 and 9 the # stays inside the quotes and is not treated as a
comment, so the value comes out as a # b. Node resolves the quote first, then looks for
comments. Sensible — but being sensible is not the same as being documented.
Then there is ;, systemd's second comment marker. systemd skips the ; yorum
line on line 23 (its docs say so) and Node skips it too (Node's docs do not mention
;; the measurement shows the line being silently passed over). But docker run rejects
the whole file:
docker: invalid env file (p23.env): variable '; yorum' contains whitespaces.
Compose rejects it too, with a different sentence:
line 1: unexpected character ";" in variable name "; yorum"
The shell does not escape either — it fails hardest. A ; is a syntax error in
the shell, so sourcing stops right there and none of the lines behind it are
read:
$ /bin/sh -c 'set -a; . ./semi-multi.env; echo "A=[$A] B=[$B] C=[$C]"'
semi-multi.env: line 1: syntax error near unexpected token `;'
A=[] B=[] C=[]
Node reads the same file as A=bir B=iki C=uc. So a single comment line takes the
whole file down in three of the five tools, while two of them do not care.
export: one ignores it, one shouts, one leaves a single log line
Line 5 is my favourite, because it shows three different failure modes at once.
The line export EXPORTED=merbay is treated as follows:
-
Node ignores the keyword; its docs state plainly that an
exportprefix before a key is ignored. -
The shell sees a perfectly ordinary line and gives
merbay. -
Compose also reads
merbay. -
docker run rejects the entire file:
variable 'export EXPORTED' contains whitespaces. My combined fourteen-variable file was never read because of this one line; not a single one of the fourteen reached the container. -
systemd drops the line but does not stop the service. The unit starts
normally, there is no variable called
EXPORTED, and all that is left behind is one journal line — which names the file, and is logged at error level, not as a warning:Ignoring invalid environment assignment 'export EXPORTED=merbay': /path/a.env.
If you ask me which is more dangerous, it is not the one that shouts. When
docker run rejects the file, the container never starts and you know within five
seconds. When systemd skips the line, the service starts, runs, writes one line to
the journal, and blows up three days later somewhere with a key that was never
there — and it takes that explosion before anyone thinks to go looking for the
line.
There is a further split inside systemd itself: the warning only appears when the
key name is invalid. For COLON: merbay and the bare BAREVAR line — lines
with no = at all — not a word reaches the journal; the unit log came back
completely empty. Its documentation already says such lines are ignored, and
ignoring makes no sound.
Where $BASE expands and where a backtick runs
Line 14 reads INTERP=$BASE. The shell and Compose turn it into merbay —
Compose's docs state that interpolation applies to unquoted and double-quoted
values. Node, docker run and systemd leave the literal string $BASE alone. Two
of five expand the variable, three do not; same file.
Line 16 genuinely stopped me:
RCE=`id -u`
Node reads the value as id -u, because in Node's spec a backtick is a quote
character and gets stripped. systemd, docker run and Compose treat it as an
ordinary character and give `id -u`. The shell executes it and returns
501 — my user id.
The asymmetry matters: sourceing a .env file is not reading it, it is
executing it. If someone who can write to that file puts a backtick in a value,
they have run code in every script that loads it with set -a; . .env. The same
line is text to one parser and a command to the shell. I wrote earlier about what
environment variables fail to keep secret in
the systemd Credentials article;
the problem here goes beyond confidentiality — the file itself is an execution
surface.
Escapes, line endings and whitespace
Line 18 is ESCAPE="a\tb": a backslash, a t, four separate characters. Of the
five tools, only Compose turns it into a real tab; its docs say \n, \r,
\t and \\ are supported inside double quotes. systemd's docs say the opposite:
inside double quotes, if a backslash precedes anything other than ", `,
\ or $, both the backslash and that character are preserved verbatim. The
measurement confirms the documentation.
Line 19 ends with a backslash. systemd and the shell treat it as a line
continuation and produce biriki; Node, docker run and Compose give bir\ and
discard the second line.
Line 17 has a real newline inside the quotes. Four of them understand the
multi-line value. docker run reads line by line, so it cuts the value at "bir
— half a value, with the opening quote still attached.
Whitespace is messier still. SPACED = merbay is read as merbay by Node,
Compose and systemd; docker run rejects the file (variable 'SPACED ' contains); the shell defines no variable at all, because to the shell that
whitespaces
line is an attempt to run a command called SPACED. Line 12
(SPACEQ= "merbay") is sneakier in the shell: that is a temporary assignment
prefixed to a command, so SPACEQ never enters the environment.
And line 13, the trailing space. Four of them trim it; docker run does not —
merbay␠␠␠. The place where you notice three spaces welded to the end of a token
is, unfortunately, the line where signature verification fails.
The bare VAR line
Line 21 is just BAREVAR, no =. Docker's docs define this explicitly: the value
is taken from the local environment. Node and the shell show the same value in the
table, but there the file contributes nothing — the value is plain inheritance.
I had filed Compose under that passive group too, until a control run proved me
wrong. With an env_file that never mentions BAREVAR, the key never reaches the
container at all; the moment the line goes into the file, the host value arrives:
bare.env (BAREVAR line present) -> BAREVAR=HOST-DEGERI
nobare.env (line absent) -> BAREVAR undefined
So Compose actively resolves a bare key against the host environment, exactly like
docker run. systemd ignores the line, because its docs say lines without an =
separator are skipped.
This trap cuts both ways. On the carrying side — docker run and Compose — a
stray bare PATH or AWS_SECRET_ACCESS_KEY line moves a value from the host
environment into the container; you can leak it without writing a value at all.
That is not a theoretical warning for me: /etc/mustafaerbay.env is read by
Compose on this server, so the leaking path is the path I run in production. On
the systemd side the same line does nothing, which is how it eats your afternoon
with "I wrote it in the file but the service can't see it".
My own template: 20 of 25 blank keys came back filled
After this measurement it would have been cowardly not to look at my own repo. The
setup template for this blog is .env.example, and the file itself says:
cp .env.example .env. Then scripts/setup.sh copies it verbatim:
if [ ! -f "/etc/$SLUG.env" ]; then
if [ -f .env ]; then
sudo cp .env "/etc/$SLUG.env"; ok ".env values copied into /etc/$SLUG.env"
else
sudo cp .env.example "/etc/$SLUG.env"
The second branch is even more direct: with no .env at all, the script installs
the template itself at the live path.
The template has 29 assignment lines. Four carry a default value; 25 were
deliberately left blank by the author — that is, me — each with an inline comment
explaining where to get the value:
GEMINI_API_KEY= # → https://aistudio.google.com/apikey
SMTP_PORT=465 # 465 = SSL (recommended) · 587 = STARTTLS
MAX_POSTS_PER_DAY=6 # Maximum posts per day
Line 21 of that template also carries my own promise: optional lines left blank
silently disable that feature. I fed the template to all five parsers:
| parser | of the 25 keys that should be empty, non-empty | the 4 lines with defaults |
|---|---|---|
| node --env-file | 0 | clean |
| sh | 0 | clean |
| docker run | 20 | comment welded to the value |
| compose | 20 | clean |
| systemd | 20 | comment welded to the value |
GEMINI_API_KEY's value in the template is 12 spaces, then a comment. In three
parsers it is not empty: docker run returns those 12 spaces plus
# → https://aistudio.google.com/apikey, while Compose and systemd trim the
leading spaces and return the same comment. Twenty keys behave this way; the other
five have no comment next to them, so they come through clean.
Why this matters: a "is the key set?" check in code is usually an empty-string
check. The value is no longer an empty string, it is a comment containing a URL.
The feature does not get "silently disabled" — it looks enabled.
Compose appearing on that list is interesting in its own right. Its docs treat a
space before the # as sufficient for an inline comment, and there is a space
here; but since the value is otherwise empty, trimming the leading spaces makes
# the first character of the value and the comment rule never fires. This one is not in the docs
anywhere I could find — only in the output.
What happened on the live file
So did this hit the site? No — and the reason embarrassed me slightly: the live
/etc/mustafaerbay.env was never copied from the template, it was written by
hand. 22 lines, 14 assignments, zero assignments carrying an inline comment.
But running that same file through the twenty-four-line filter above matched two
lines: SMTP_PASS and MAIL_FROM. Both values are quoted. The measurement here used
MAIL_FROM, which is not a secret: 44 characters in the raw line, 42 in the value
systemd actually reads. It stripped the quotes, two characters. (I am not printing
SMTP_PASS's length; the character count of a password is not something to share
either.)
Today this works, because the two tools reading that file (the systemd units and
Compose) happen to be two tools that strip quotes. The code that consumes
SMTP_PASS also runs under units carrying EnvironmentFile=. In other words, the
system stands because two parsers happen to agree on this particular point.
Add a third reader — one docker run --env-file /etc/mustafaerbay.env line in a
deploy script — and the same password is delivered with two extra characters. The
error will read "SMTP authentication failed", not "your value has two quotes in
it".
Once parsing is done, precedence starts
So far this has all been "what the file says and what the tool reads". There is a
second axis, and it splits just as badly: when the same key is defined both in the
environment and in the file, which one wins? A one-line file
(PREC=dosyadan) against PREC=ortamdan in the environment answers it:
| parser | winner |
|---|---|
| node --env-file | the environment |
| sh | the file |
| docker run --env-file | the file |
| compose env_file | the file |
| systemd EnvironmentFile | the file |
Node stands alone again, and in the most annoying place: its docs state it plainly
(if the same variable is set in the environment and in the file, the environment
wins), but Node is the tool you use for local development. An
export SMTP_PORT=25 you forgot about in your shell quietly neutralises the line
in .env — you fix the file and nothing changes.
On the systemd side the source order is documented and matches the measurement
exactly: the manager's environment, then Environment=, then EnvironmentFile=,
with the later source winning. Putting Environment= after EnvironmentFile= in
the unit does not change it either; I tried the reversed order and the file still
won.
docker run has one exception: a bare -e PREC makes the host value override the
one from the file. Two paths in the same command, racing each other.
The lowest common denominator
The only way to satisfy five parsers at once is to stay inside the intersection
they all support. The rules this measurement left me with:
-
Don't use quotes. If the value has no spaces, quotes buy you nothing and
cost you something in
docker run. -
Don't write inline comments. Put the comment on its own line above the key,
starting with
#. -
Don't comment with
;. Only systemd and Node get through;docker runand Compose reject the file, and the shell stops on a syntax error. -
Don't write
export.docker runrejects the file; systemd skips the line and leaves one log entry behind. -
No spaces around
=, and no trailing space at the end of the line. -
Don't rely on
$VAR,$(...)or backticks.$VARis expanded by two (the shell and Compose),$(...)and backticks are executed only by the shell, and Node strips backticks thinking they are quotes. Write the value out. - No multi-line values. Certificates and the like belong in systemd credentials or a mounted secret file, not here.
-
Don't use the
:separator. It only works in Compose. -
Don't write a bare
VARline. Two tools carry it from the host (docker runand Compose), one ignores it.
Compose does give you an escape hatch: set format: raw on an env_file entry
and Compose stops interpreting the value, passing quotes and $ through as-is
(Compose 2.30 and later). If you only have one parser, that is good news; with
five, simplifying the file is still cheaper.
And the sturdiest path is not to copy the template but to derive from a single
source: keep the human-readable file separate and generate the tool-facing file
from it. For now I am starting somewhere more modest — a check script. All nine
rules above fit in one grep:
#!/bin/sh
# Finds lines where the five parsers may disagree.
grep -nE '^[[:space:]]*export |^;|^[A-Za-z_][A-Za-z0-9_]*$|^[A-Za-z_][A-Za-z0-9_]*[[:space:]]*:|^[A-Za-z_][A-Za-z0-9_]*[[:space:]]+=|^[A-Za-z_][A-Za-z0-9_]*=.*[#"'"'"'`$]|^[A-Za-z_][A-Za-z0-9_]*=.*[[:space:]\]$|\\$' "$1"
Writing a script and declaring it works is not enough, because a scanner that
finds nothing also stays quiet. I ran it over all twenty-four probe files one by
one: it caught 20 of the 20 divergent lines and raised no false alarm on any
of the 4 lines where all five agreed. On a clean file it says nothing at all. On
my own template it flagged 24 lines; on the live server's file, 2.
Conclusion
What I take from this is not a list of tools. It is this: two file formats sharing
a name are not the same format. Because .env is not a standard, every tool made
a reasonable interpretation, and five reasonable interpretations disagree on
twenty of twenty-four lines. The docs are not at fault either — all five wrote
down their behaviour; I was the one who had not read all five together.
Recognising this pattern in infrastructure pays off: when a data file has more
than one consumer, the file's real format is bounded by the strictest consumer's
rules. Instead of guessing from the docs which consumer that is, writing
twenty-four lines and measuring took me half an hour. My own file works today only
because two of its readers happen to agree, and a coincidence is not an
architectural decision.
Official Sources
- Node.js —
--env-file=file(quotes, comments, multi-line values,export) - Docker —
docker runreference: "Set environment variables (-e, --env, --env-file)" - Docker Compose —
servicesreference:env_file,format: rawand the "Env_file format" rules - Docker Compose — variable interpolation
- systemd —
systemd.execman source (v255):EnvironmentFile=parsing rules
Top comments (0)