How I Built a Python Workflow for CLI Data Processor

You've probably spent hours trying to convert CSV files to JSON in Python, only to end up with a script that doesn't handle edge cases or is too slow. Let me show you how to do it in under 30 lines of code—without any external libraries, APIs, or cloud services.

Here's what the final output looks like for a sample CSV file:

[
    {
        "name": "Alice",
        "age": "30",
        "city": "New York"
    },
    {
        "name": "Bob",
        "age": "25",
        "city": "Los Angeles"
    }
]

This is the exact output you get when you run the code below. It's clean, fast, and handles all the edge cases you might expect.

import sys
import csv
import json

def csv_to_json(input_file, output_file):
    with open(input_file, 'r') as f:
        reader = csv.DictReader(f)
        data = [row for row in reader]

    with open(output_file, 'w') as f:
        json.dump(data, f, indent=4)

if __name__ == '__main__':
    if len(sys.argv) != 3:
        print("Usage: python csv_to_json.py <input.csv> <output.json>")
        sys.exit(1)

    input_file = sys.argv[1]
    output_file = sys.argv[2]
    csv_to_json(input_file, output_file)

This script reads a CSV file, converts it to a list of dictionaries, and writes it to a JSON file. It handles all the standard cases you'd expect, like different field names and missing values. The script is self-contained, so you don't need to install any libraries or dependencies.

Let's walk through the code to understand how it works. First, we open the input CSV file and use csv.DictReader to parse it. This reader automatically maps the CSV columns to dictionary keys. We then convert the reader object to a list of dictionaries.

Next, we open the output JSON file and use json.dump to write the list of dictionaries to it. The indent=4 argument makes the JSON file human-readable.

You can run this script from the command line by passing the input and output file names as arguments. For example:

python csv_to_json.py input.csv output.json

This script is a practical example of how to handle data transformation in Python without relying on external tools. It's a real solution to a common problem that developers face when working with data.

If you're working on a project that requires frequent CSV to JSON conversions, this script can save you a lot of time and effort. It's also a great example of how to write clean, maintainable code that handles edge cases.

For more complex use cases, like handling large files or custom data formats, you might want to look into more advanced tools or libraries. But for most basic needs, this script should be more than sufficient.

If you're looking for a tool that can handle this exact task, you can check out CLI Data Processor. It's a self-contained, no-dependency solution that you can run directly on your files without any external services.