loading...

A shell and GNU awk oneliner

jmcp profile image James McPherson ・1 min read

I'm updating a test suite for $work, and need to handle input data with differing numbers of columns. The input is CSV files, which is fairly friendly. The number of files I've got to test my code against is extensive and I wanted to quickly figure out how many of each type I have.

Enter the one-liner:

$ for f in *csv; do awk -F, '{END print ARGV[ARGC-1], ":\t", NF}'
file1:      10
file2:      11
file3:      14
file4:      10
....
file100:     14
$

For each file in the list, when GNU awk has read it all in and parsed it using the , as a field delimiter (the END), we then print the filename (ARGV[ARGC-1]), and the number of fields NF. If I wanted to print the count of the number of records (lines) in the file, I could add NR to the arguments to print.

I hope you found this useful.

If you want to explore GNU Awk more, the documentation is at https://www.gnu.org/software/gawk/manual

Posted on by:

jmcp profile

James McPherson

@jmcp

Data wrangler, software engineer, systems programmer, cyclist. Unix (mostly Solaris) for aeons. I talk C, Python, SQL and Makefiles. I take big pictures and create architecture to make them a reality.

Discussion

pic
Editor guide