DEV Community

Cover image for `xargs rm` was aimed at one file and hit a different one
techen spectre
techen spectre

Posted on AI-assisted

`xargs rm` was aimed at one file and hit a different one

There are two files here, and you want to delete the meeting notes:

$ ls
meeting notes.txt
notes.txt
Enter fullscreen mode Exit fullscreen mode

So you'd write this:

ls | grep meeting | xargs rm
Enter fullscreen mode Exit fullscreen mode

Before running it, ask xargs what command it's about to build:

$ ls | grep meeting | xargs echo rm
rm meeting notes.txt
Enter fullscreen mode Exit fullscreen mode

That's two arguments: meeting, which doesn't exist, and notes.txt, which is the other file. Running it would leave the meeting notes alone and delete notes.txt.

The mental model most people have is wrong

People tend to read | xargs cmd as "send the input to cmd". That's what a plain pipe does:

$ echo notes.txt | cat
notes.txt

$ echo notes.txt | xargs cat
important
Enter fullscreen mode Exit fullscreen mode

The pipe gave cat the text notes.txt. xargs gave cat the argument notes.txt, so cat opened the file.

xargs does three things:

  1. Reads text from stdin.
  2. Splits it into words at spaces, tabs and newlines. Quotes and backslashes are treated specially too.
  3. Builds one command line from as many of those words as fit, and runs it.

Everything people find confusing about xargs follows from those three steps.

Every common surprise, explained by that model

It splits on spaces, not lines. This is the opening example. meeting notes.txt is one line but two words. Use NUL separators so names can't be split:

$ find . -name 'meeting*' -print0 | xargs -0 printf '[%s]\n'
[./meeting notes.txt]
Enter fullscreen mode Exit fullscreen mode

With GNU xargs, -d '\n' also works when you only have one name per line:

$ ls | grep meeting | xargs -d '\n' printf '[%s]\n'
[meeting notes.txt]
Enter fullscreen mode Exit fullscreen mode

Quotes are special. An apostrophe in a filename stops it with an error:

$ echo "it's.txt" | xargs echo
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
Enter fullscreen mode Exit fullscreen mode

It runs the command once, not once per line. This is the most-asked xargs question on Stack Overflow, with over 500 votes:

$ printf 'a\nb\nc\n' | xargs echo run:
run: a b c

$ printf 'a\nb\nc\n' | xargs -n1 echo run:
run: a
run: b
run: c
Enter fullscreen mode Exit fullscreen mode

xargs packs the words into as few command lines as it can. Use -n1 for one argument per run, or -I{} to place each line exactly where you want it.

Empty input still runs the command. With GNU xargs, no input means the command runs once with no arguments:

$ find . -name '*.orig' | xargs ls -l
total 0
-rw-r--r--. 1 user user 0 Nov  7 00:06 important.txt
Enter fullscreen mode Exit fullscreen mode

Nothing matched, so ls -l ran with no arguments and listed the current directory. -r (--no-run-if-empty) skips the run:

$ find . -name '*.orig' | xargs -r ls -l
$
Enter fullscreen mode Exit fullscreen mode

> doesn't go where you think. The shell handles redirection before xargs even starts:

$ printf 'x\ny\n' | xargs -I{} echo {} > {}.out
$ ls
{}.out
$ cat {}.out
x
y
Enter fullscreen mode Exit fullscreen mode

That created one file, literally named {}.out, holding all of xargs's output. To redirect per item, give each run its own shell:

$ printf 'x\ny\n' | xargs -I{} sh -c 'echo "$1" > "$1.out"' _ {}
$ ls
x.out
y.out
Enter fullscreen mode Exit fullscreen mode

Two habits that prevent all of this

See the command before you run it. Put echo in front of the command for a dry run, as in the opening. Or use -t, which prints each command to stderr as it runs:

$ printf 'a\nb\n' | xargs -t echo
echo a b
a b
Enter fullscreen mode Exit fullscreen mode

Often you don't need xargs. If the list comes from find, -exec ... {} + batches arguments the same way, and it never re-splits names:

$ find . -name 'meeting*' -exec printf '[%s]\n' {} +
[./meeting notes.txt]
Enter fullscreen mode Exit fullscreen mode

Which of these has bitten you: spaces, empty input or the >?

Top comments (0)