DEV Community

Fun with Bash

Chris James on August 12, 2018

This was originally posted on my blog a while ago I was given a fairly mundane task at work and when this happens I try and find ways to make it i...

Read full post

Mathieu PATUREL • Aug 12 '18 • Edited

Cool command! You could remove the head bit though with awk.

awk 'condition { command }'

By default the condition is NUL, which matches every line (so the command is run on every line). You have access to different variable (in both the condition and the command I think), one of them being NR for the line number.

So, if you set the condition to NR!=1, it'll run on every line different than 1 (which is the first one).

So, your command can be shortened from:

awk -F , '{print $1, $3-$2}' data.csv | tail -n +2 | sort -r -n --key=2 | column -t

awk -F , 'NR!=1{print $1, $3-$2}' data.csv | sort -r -n --key=2 | column -t

:smile:

I learned a bunch otherwise, so thanks!

Chris James • Aug 12 '18

Thanks for the improvements!

Vlastimil Pospichal • Aug 12 '18

data.csv

name, usage (mb), allocated (mb), CPU %, containers
dispatcher, 150, 512, 40, 10
assembler, 175, 512, 75, 10
matcher, 85, 512, 15, 10
"user, profile", 128, 512, 40, 5
profile-search, 220, 512, 80, 10

column -s, -t < data.csv

name             usage (mb)   allocated (mb)   CPU %   containers
dispatcher       150          512              40      10
assembler        175          512              75      10
matcher          85           512              15      10
"user            profile"     128              512     40           5
profile-search   220          512              80      10

Not applicable to general CSV.

Ben Sinclair • Aug 12 '18

I've never used column before. That's pretty cool.
As far as the WTFness of it all goes, if you split it up and assign variables with nice names it'll be easy enough to understand. There's something about shell scripts though that make people write things as tersely as possible. I'd end up with something like that and refer back to it in my history if I needed it again soon after, but if I made it into a script I'd either comment the hell out of it or split it into parts, I think.

Bugfix:

tail -n X prints the last X lines, the plus inverts it so its the first X lines.

if it did that, it'd be head :) The plus means take the offset from the start of the file instead of the end.

Khillo81 • Sep 18 '19

May I suggest csvkit as a means to work with CSV data at the command line? It has several nifty tools to handle CSV files including getting rid of the column headers, splitting and merging several tables by columns, and even has ways of converting Excel xlsx tables to csv. I would recommend giving it a try if you have to deal with these types of files at the command line.