DEV Community

Bobo
Bobo

Posted on

Command Line CSV Processing Tutorial: Merge, Clean, and Convert from Terminal

Command Line CSV Processing

When you have dozens of CSV files to merge, clean, and convert, doing it manually in Excel is painful.

Here is how to do it from the terminal.

Merge Multiple CSV Files

toolkit csv merge sales_*.csv -o merged.csv
Enter fullscreen mode Exit fullscreen mode

Clean Duplicates

toolkit csv clean merged.csv --deduplicate -o clean.csv
Enter fullscreen mode Exit fullscreen mode

Convert to JSON

toolkit json convert data.csv -o data.json
Enter fullscreen mode Exit fullscreen mode

Complete Pipeline

Create pipeline.json:

{
  "name": "Monthly Report",
  "steps": [
    { "tool": "csv", "args": ["merge", "sales_*.csv", "-o", "merged.csv"] },
    { "tool": "csv", "args": ["clean", "merged.csv", "--deduplicate", "-o", "clean.csv"] },
    { "tool": "json", "args": ["convert", "clean.csv", "-o", "report.json"] }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Run: toolkit pipeline pipeline.json

Why CLI?

  • 10x faster than Excel
  • Works with files Excel cannot open
  • Repeatable and scriptable
  • Can run on a schedule

Get It

Install: npm install -g cli-toolkit-pro

Free (file/text): Always free
Pro ($0.99): CSV, JSON, Image, PDF, Pipeline

https://bobotempes.gumroad.com/l/nmcqjr


More Resources

  • 🛠️ My tools & open-source projects: GitHub
  • 📝 Blog & updates: TucaoWall
  • ☁️ Deploy your own cloud server and get credit: DigitalOcean
  • 🐙 CLI Toolkit Pro (bundle of all my CLI tools): npm
  • 💼 Hire me on Freelancer: a0982

Top comments (0)