Never Name a zsh Loop Variable path: Three Deploy-Verification Traps That Looked Like Outages

In one day of post-deploy verification I hit three failures that all looked like "production is broken" and none of which were: zsh's tied path array destroying PATH, percent-encoded URLs compared against raw ones, and plain propagation delay.

I run a static site generator, and after each deploy a zsh script mechanically verifies that "all 10 public surfaces return HTTP 200, and their contents match the local build output (dist)".

Running that verification for a day, I hit three traps of completely different natures: a bug in the verification script, a bug in the comparison logic, and propagation delay on the delivery side. Every one of them could have led straight to the wrong conclusion — "production is broken" — so I am writing them down.

Trap 1: Using zsh's reserved path array as a loop variable

I had named the loop variable of a hash-comparison loop path without thinking.

# the broken code
for path in dist/index.html dist/rss.xml; do
  shasum -a 256 "$path"
done

Here is what you get:

zsh:1: command not found: shasum
zsh:1: command not found: shasum

The files exist, but now the command cannot be found.

The cause is zsh's tied array. In zsh, path is typed and linked to PATH; rewriting one rewrites the other.

$ typeset -p path
typeset -aT PATH path=( /opt/homebrew/bin /usr/local/bin /usr/bin /bin ... )

The -T in typeset -aT PATH path is what does it. In other words, for path in ... is not an assignment to an ordinary loop variable — it is a rewrite of PATH. Printing $PATH inside the loop made it obvious:

path=dist/index.html
PATH=dist/index.html

PATH now has exactly one entry, dist/index.html, so every subsequent command lookup fails. The fix is just renaming the variable.

# the fixed code
for file_path in dist/index.html dist/rss.xml; do
  shasum -a 256 "$file_path"
done

Command lookup survives and each file's SHA-256 prints normally.

path is not the only tied name. cdpath, fpath, manpath, module_path and others are also linked to their lowercase array counterparts. In zsh scripts you can avoid all of this simply by not using bare lowercase common nouns as loop variables (use file_path, target, url, and so on).

If you use one inside a function, declaring it local closes the scope so that PATH is restored after the function returns. But PATH is still broken while the function runs, so it is not a real workaround.

check() {
  local path                      # PATH outside the function is protected
  for path in a b; do :; done     # but you cannot call commands in here
}

The nasty part is that the error message is command not found: shasum. If you start investigating it as a path or filename problem, you will take a long detour. When commands suddenly go missing in zsh, suspect the name of the loop variable you just wrote — that was the shortest route.

Trap 2: Comparing URL-encoded Japanese without decoding it

The internal-link check extracts links from the production HTML and matches them against the generated files in dist. In that comparison, URLs containing Japanese tags produced six false positives (reported as broken links) in one batch.

The hrefs in the production HTML are percent-encoded, while the local file name list was still raw Japanese. As strings they obviously do not match.

production HTML :  /tags/%E7%B1%B3%E5%9B%BD%E7%B5%8C%E6%B8%88/
local           :  /tags/米国経済/

After decoding and re-comparing, all 21 unique internal links returned HTTP 200 and were fine. What was broken was the verification script, not production.

On sites that put Japanese tags or categories into URLs, you must normalize before comparing (decode, or encode both sides the same way). When several broken links suddenly appear at once, it pays to suspect the comparison code before suspecting the site.

Trap 3: Almost classifying a post-deploy 404 as an outage

On the first check immediately after the deploy completed, I saw:

• New article page: HTTP 404
• Markdown mirror: HTTP 404
• Aggregate surfaces such as the top page and indexes: HTTP 200, but showing the old content

This is the classic look of "delivery is broken". It is tempting to panic and push again or purge caches, but I did not push again; I waited 20 seconds and re-verified, and all 10 surfaces returned HTTP 200 with SHA-256 matching the local dist. It was simply propagation delay.

If you push again in that moment, a false causality — "I fixed it, so it got fixed" — gets recorded, and you repeat the pointless operation every time after that. The right design is for the verification script to wait a fixed interval and retry after the first failure, and report only the surfaces that still fail as an incident.

What went back into the verification script

Based on those three, post-deploy verification settled into this shape:

1. Never use zsh tied-array names such as path as loop variables (make it a fixed review checklist item)
2. Always decode before comparing URLs. Treat encoding differences as "not normalized", not "does not match"
3. Do not treat the first HTTP failure as final; wait, then re-check
4. Make the pass condition "HTTP 200 and SHA-256 matching the local dist", not just "HTTP 200"

Item 4 in particular pays off. If a 200 alone is enough, the old version can keep being served and still pass; only by checking the content hash can you say it was actually deployed. In trap 3 the aggregate surfaces were exactly that — "200 with old content".

Summary

• zsh's path is a reserved array tied to PATH. Using it as a loop variable breaks command lookup, and the symptom shows up as command not found
• Comparing Japanese URLs without normalizing the encoding difference mass-produces false positives
• A 404 or stale content right after a deploy may be propagation delay. Wait and re-check before doing anything else
• Do not make "HTTP 200" the only pass condition. Deployment is only confirmed once the content hash matches

All three would have landed on the wrong conclusion — "production broke" — if left alone. It comes down to the obvious point that the verification script is itself something that needs verifying.

---

I also publish measurement records around verification and automation on ACS Developer.

Originally published in Japanese on Zenn: https://zenn.dev/acs_developer/articles/zsh-path-array-deploy-verify-pitfalls

Comments

[Alex Shev]

A useful next guardrail is making the verifier report the normalization path and retry state beside the final result. Then a failed check answers two questions at once: which surface disagreed, and whether the disagreement is content, encoding, or still-propagating delivery.