A for Loop Skipped 23 Files and Called It a Successful Backup

The backup ran every night at 2am and emailed me a green "847 files archived" summary. I'd built it, tested it against my own home directory where every file was named like report_2024.csv, watched it sail through, and shipped it. For weeks the summary said everything was fine.

Then a coworker asked me to restore a file. Q3 forecast.xlsx. It wasn't in the archive. Neither was Annual Review FINAL.pdf, or meeting notes (draft).md, or any of the other 23 files someone had named the way normal humans name files — with spaces in them. The backup had been quietly skipping the most important files on the share for a week, and the nightly email had been telling me the whole time that nothing was wrong.

Here's the line that did it:

for file in $(ls "$DIR"); do
  echo "Processing $file"
done

The problem is word splitting, and it's invisible until the day it isn't. An unquoted $(ls ...) splits its output on every space, so Q3 forecast.xlsx doesn't arrive as one filename — it arrives as two iterations, Q3 and forecast.xlsx. Neither one exists on disk. The loop tries both, finds nothing, shrugs, and moves on without a single error. The script "succeeds" because from its point of view it did exactly what it was told.

I've now got two habits burned in, and I don't write a loop without both. The first: glob the directory, never parse ls.

for file in "$DIR"/*; do
  [[ -e "$file" ]] || continue   # the glob yields a literal '*' on an empty dir
  echo "Processing $file"
done

"$DIR"/* hands you each real path as a single unit. No subshell, no string to re-split, no ls output to misread. The [[ -e "$file" ]] || continue guard covers the one quirk of globbing: when a directory is empty, * expands to the literal character *, and without the guard you'd try to process a file named *.

The second habit: quote every expansion, every time. "$file", never $file. The day a filename has a space in it, the unquoted version splits into two arguments and your command operates on paths that were never there.

Arrays follow the exact same rule, and the distinction that matters is [@] versus [*]:

servers=("web-01" "db-prod 02" "cache-03")

for host in "${servers[@]}"; do
  echo "Connecting to $host"   # three clean iterations, the space survives
done

"${servers[@]}" in double quotes gives you one word per element — db-prod 02 stays whole. "${servers[*]}" joins everything into a single string and is almost never what you want in a loop. Drop the quotes on either and you're back to the bug that ate my backup.

When you genuinely need a counter — walking app.log.1 through app.log.9, numbering batches, counting retries — use the C-style form. Do not reach for for i in {1..$n}; brace expansion runs before variable expansion, so $n never gets substituted and you loop once over the literal text {1..$n}. Ask me how I know.

for ((i = 1; i <= MAX_ROTATED; i++)); do
  log="$LOG_DIR/app.log.$i"
  [[ -f "$log" ]] || continue
  echo "Scanning $log"
done

And reading a file line by line — for line in $(cat file) is wrong in two directions at once. It splits on whitespace instead of newlines, so a line with spaces becomes several iterations and blank lines vanish, and it globs, so a line containing * expands to filenames in your current directory. The form that survives real files:

while IFS= read -r line; do
  echo "Line: $line"
done < "$INPUT_FILE"

IFS= stops bash from trimming leading and trailing whitespace. -r stops it from eating backslashes. One line per iteration, exactly as written.

The production script I keep on the site ties all of this together — globs the directory, quotes every use, counts successes and failures separately, and exits non-zero if anything failed. That last part is the one people skip, and it's the one that matters: a loop that processes files but always exits 0 is how you end up with a nightly email that says 847 when the real number is 824.

The cost of my mistake was a week of bad backups and the specific discomfort of a coworker finding the gap before my own tooling did. I'm not interested in repeating that, so the quoting is muscle memory now. Yours might as well be too.

Full script with all four loop forms — files, arrays, counters, and line-by-line reads — production-ready and ShellCheck-clean: https://bashsnippets.xyz/snippets/bash-for-loop-examples

If your loops are the thing crashing scripts, the Bash Error Handling snippet pairs with this one, and the rest of the library is at https://bashsnippets.xyz