Shell commands for simple tasks of processing CSV file - Linux Tips

Paste

Use paste to concat 2 csv files:

$ cat students.csv
Name,Grad Year
Alice,2024
Bob,2022
Chris,2023

$ cat score.csv
Name,GPA
Alice,B+
Bob,A
Chris,A-

# concat 2 csv files
$ paste -d "," students.csv score.csv
Name,Grad Year,Name,GPA
Alice,2024,Alice,B+
Bob,2022,Bob,A
Chris,2023,Chris,A-

Cut

Use cut to get designated columnes:

# Use -f 1,2,4 to get only 1st, 2nd and 4th column
$ paste -d "," students.csv score.csv | cut -d , -f 1,2,4
Name,Grad Year,GPA
Alice,2024,B+
Bob,2022,A
Chris,2023,A-

Sort

Use sort to sort specified field in CSV file:

$ cat students.csv
Alice,85,[e-mail]
Bob,92,[e-mail]
Chris,60,[e-mail]

# now sort by score in descending order
# -t: split by
# -k: field number after split
# -n: compare field as number, not string
# -r: in reverse order
$ sort -t , -k 2 -nr students.csv
Bob,92,[e-mail]
Alice,85,[e-mail]
Chris,60,[e-mail]

Awk

Use awk to get fields in CSV file:

$ cat students.csv
Alice,85,[e-mail]
Bob,92,[e-mail]
Chris,60,[e-mail]

# use `awk` to split each line and get 2nd field
$ cat students.csv | awk -F, '{print $2}'
85
92
60

# pipe with `sort` to get the highest score
$ cat students.csv | awk -F, '{print $2}' | sort | tail -1
92