XARGS only makes your life easier

One command-line tool that would make your life easier is the xargs command.

So, what is xargs?

xargs is part of the Findutil package of the GNU project which builds and executes commands using arguments read from standard inputs.

What are Standard Inputs in Linux?

Standard inputs are just a bunch of text on the command line either read from a file or keyboard that makes up a Linux command. So, let us examine the seq command here which prints out in a new line a sequence of numbers. I recommend you create a new directory in your home directory preferably for this tutorial. Let’s play!!!💃

$mkdir tutorial
$seq 1

The arguments read for the standard input of this command is just 1.
While for this command:

$seq 3

The arguments read for the standard input of this command above are 1, 2, and 3.

Now that you understand what standard input is, you might want to supply it as an argument to a new command. Maybe because you don’t want to presume the arguments being read beforehand or maybe because the argument list is just too long and typing would be a waste of time. That is where xargs comes in.
We can combine these arguments using xargs with all commands that there is on Linux with this syntax:

command | xargs command

Let us try xargs with the echo command:

$seq 1 | xargs echo
1
$seq 3 | xargs echo
1 2 3

And the default of the xargs command is echo anyway, so you don’t need to include it:

$seq 1 | xargs 
1
$seq 3 | xargs 
1 2 3

And the touch and mkdir command:

$seq 1 | xargs -I {} touch version{}.txt
$ls
version1.txt
$seq 2 | xargs -I {} mkdir v{}
$ls
v1  v2  version1.txt

In the 2 commands above, we introduce a new option –I which replaces the character {} as a placeholder for the arguments read from the standard input and even included more prefix and suffix characters(v, version and.txt). And the placeholder doesn’t have to be {}, it could be % or any character you deem fit. In the first command, we used this technique to create a text file named version1.txt. And in the second command, two directories named v1 and v2 are created.
Standard input can also come from a file. Let us edit the content of version1.txt and put some content in it. Let us put Google, Facebook, and Netflix as new lines.
$nano version1.txt

Save with: Ctrl+x+y

Let us see whether our server can reach the companies outlined in the text file and maybe they are up:

$cat version1.txt| xargs -t -I {} ping -c 2 {}.com
ping -c 2 google.com 
PING google.com (216.58.223.238) 56(84) bytes of data.
64 bytes from los02s04-in-f14.1e100.net (216.58.223.238): icmp_seq=1 ttl=117 time=1.56 ms
64 bytes from los02s04-in-f14.1e100.net (216.58.223.238): icmp_seq=2 ttl=117 time=1.63 ms

--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1004ms
rtt min/avg/max/mdev = 1.560/1.596/1.633/0.054 ms
ping -c 2 facebook.com 
PING facebook.com (102.132.101.35) 56(84) bytes of data.
64 bytes from edge-star-mini-shv-01-los2.facebook.com (102.132.101.35): icmp_seq=1 ttl=55 time=1.34 ms
64 bytes from edge-star-mini-shv-01-los2.facebook.com (102.132.101.35): icmp_seq=2 ttl=55 time=1.48 ms

--- facebook.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.342/1.412/1.483/0.079 ms
ping -c 2 netflix.com 
PING netflix.com (54.246.79.9) 56(84) bytes of data.

--- netflix.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1008ms

In this command, we use the file named version1.txt as an argument for standard input for the ping command. We also introduced the -t option for xargs which shows the command to execute before executing.
Let us do a long listing of the files and directories we have created thus far using xargs on a single line:

$ls -l| xargs -t 
echo total 12 drwxrwxr-x 2 crmdev crmdev 4096 Mar 5 14:18 v1 drwxrwxr-x 2 crmdev crmdev 4096 Mar 5 14:18 v2 -rw-rw-r-- 1 crmdev crmdev 24 Mar 5 14:45 version1.txt 
total 12 drwxrwxr-x 2 crmdev crmdev 4096 Mar 5 14:18 v1 drwxrwxr-x 2 crmdev crmdev 4096 Mar 5 14:18 v2 -rw-rw-r-- 1 crmdev crmdev 24 Mar 5 14:45 version1.txt

However, things could get messy with xargs though when what is being read from standard input are also messy. What do I mean messy? Well, when you have standard inputs maybe filenames that result in arguments that have delimiters such as spaces and newlines, things start to work strangely. Take, for example, we create a file having a whitespace called version 1.txt and we run xargs on it, using the find command to find all files with the type .txt in the current directory:

$touch version\ 1.txt
$ls
v1  v2  version 1.txt  version1.txt
$find . -iname '*.txt' -print0 | xargs -t ls -l
xargs: WARNING: a NUL character occurred in the input.  It cannot be passed through in the argument list.  Did you mean to use the --null option?

Arggh😠😠😠! xargs!!! What does this mean, right? Yes. xargs won’t work well because it treats an embedded space as a delimiter which makes the command interpret each space-separated word as a whole different argument. So, what do we do? To avoid this, we can simply use the –null or-0 option. Or press yes to the prompt shown.

$find . -iname '*.txt' -print0 | xargs --null -t ls -l
ls -l ./version1.txt ./version 1.txt 
-rw-rw-r-- 1 crmdev crmdev  0 Mar  5 15:55 ./version 1.txt
-rw-rw-r-- 1 crmdev crmdev 24 Mar  5 14:45 ./version1.txt

The problem seems to be all gone!

We have 2 files and 2 directories already now created in our playground directory. How about if I want to remove them all at once depending on the file type.

NB: xargs is a powerful command and you should use it carefully, especially with other powerful commands such as rm.
Let us use find and exec to delete the text files we created in the playground directory called tutorial and calculate the time taken to execute:

$time find . -iname '*.txt' -exec rm {} \;

real    0m0.058s
user    0m0.000s
sys     0m0.000s
$ls
v1  v2

Let us re-create the files and use xargs to delete this time:

$seq 1 | xargs -I {} touch version{}.txt
$ls
v1  v2  version1.txt
$time find . -iname '*.txt' | xargs rm

real    0m0.006s
user    0m0.000s
sys     0m0.000s

The differences seem to be clear. Asides from xargs being faster, find seems to make our life harder with typing a longer command and a complex syntax.

How about if we want to break the execution and want our commands to execute based on the number of arguments we desire per time. The immediate use-case for this is the fact that the CLI has a word limit it can take, so you don’t want a large number of arguments that may mess up your command and shrink it, and so you may want to break those long commands to smaller chunks:

$ls
v1  v2
$ls | xargs -t
echo v1 v2 
v1 v2
$ls | xargs -t -n 1
echo v1 
v1
echo v2

Here, the –n option is used to specify that it should run one argument per time. And it ran recursively after specifying the option.

You can confirm your word limit on your command line by using the –show-limits option with xargs. Let us try it with the long listing of our files again:

$ls -l| xargs --show-limits 
Your environment variables take up 2788 bytes
POSIX upper limit on argument length (this system): 2092316
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2089528
Size of command buffer we are actually using: 131072
Maximum parallelism (--max-procs must be no greater): 2147483647
total 12 drwxrwxr-x 2 crmdev crmdev 4096 Mar 5 14:18 v1 drwxrwxr-x 2 crmdev crmdev 4096 Mar 5 14:18 v2 -rw-rw-r-- 1 crmdev crmdev 24 Mar 5 14:45 version1.txt

So yeah…a world of unlimited arguments is a façade😢. And in this example, our command length is 131,072 of 2,089,528 bytes. So, we are doing well.

It is evident that xargs can make our life easier with the command line and we can spend better time doing more productive work instead. Right?

Thank you for reading!😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁