I did an xargs!

#xargs #unix #commandline

A fundamental thing you must know is that there is not a canonical set of knowledge for ... most anything, really, but here, for Linux/Unix expertise. There have been studies. We all have gaps in our knowledge. I am sure there is someone with substantial knowledge of the linked-lists underlying file system representation but is unaware that ls -lh put file sizes in a human-readable form. (If that's you, you're welcome.)

For me, one of my long-standing knowledge gaps has been xargs. I've seen it for decades, all find . | xargs <something> and I've copied and pasted those commands, but the knowledge never stayed.

segue

For the last month and a half, an API we use has not been working, meaning we couldn't update data into it. But that bug has been fixed, and now I have hundreds of updates that have been requested of me.

I have an API tool that allows me to get a whole month's worth of data out of the API and into columns, and this time I remembered awk is the one I wanted (sed and awk being two other gaps in my Unix knowledge, but that's another column), getting me:

1sf701vdlahb
3lm63b58riej
3o4rhtaxe8lx
b5hqvs6tuqnm
eoirmijclnl1
uihc1q3yzkvm
vq2uskrrlb5h
w6ulq3sdvyir
wxk3bbdpwdx3
y2npmn6yjkcx
zepz3n2e2to7
zk08o7dysokt
...

(And no, that's just random strings, but bear with me.)

So, I needed to run api_tool -status done -r for all of those, and thought through the pain of all the other options. List of IDs into a file, and then ... bash for loop? add api_tool ... to every line and sh id_list.txt?

And then I remembered xargs, and a half-dozen tabs were opened.

xargs has a lot of flags, so there's a LOT more to it than what I'm using here, but what I used was api_tool -q not_done -f 2019-06-01 | awk "{print $1}" | xargs -n 1 api_tool -status done -r.

This awk command was right the first time (yay me!) but I ran the xargs against echo a few times before I got it right. -n <number> determines the number of entries it pulls from the list of entries at once (we just want the one) and appends it to the end of the list, so <whatever> | xargs -n 1 <command> becomes <command> <first entry> and so on.

This saved me a whole lot of pointless typing, and I'm so glad.

moral

The point is that you should recognize when you have gaps, exploit opportunities to learn things, and record these things so that you (and others) can use them later.

And, xargs is cool, yo.