Fix Line-by-Line Text File Diffs Without GUIs or APIs

When working with text files in a CI/CD pipeline or version control system, the need to compare files line by line is common. Whether you're auditing changes in a config file, tracking updates in a JSON log, or validating data in a CSV, the ability to quickly and reliably identify differences is crucial. However, many developers end up using GUI-based diff tools or APIs that are either too slow, too complex, or require external dependencies. This article walks you through a practical technique for comparing text files line by line using a lightweight, self-contained Python CLI tool — and how it can be a game-changer in your workflow.

The core idea is to use a simple, deterministic algorithm to compare two text files and output a structured diff. This is not only useful for manual review but also for automating checks in scripts or CI pipelines. The tool we're focusing on, TextDiffTool, does exactly this — it compares two text files and highlights differences line by line.

Let's start by understanding how to use the tool. First, you'll need to install it. The tool is self-contained and requires Python 3. You can install it using pip:

pip install textdifftool

Once installed, you can use it from the command line with the following syntax:

textdifftool --input input.json --output output.json --verbose

This command tells the tool to read from input.json, write results to output.json, and print progress to stderr. The tool is designed to handle malformed or empty input gracefully, ensuring it doesn't crash on real-world files.

Now, let's look at how the tool processes the input. Here's a simplified version of the core logic:

def process_diff(input_file, output_file, verbose):
    with open(input_file, 'r') as f:
        lines = f.readlines()

    # Simulate diff logic
    diff_result = {
        "differences": [],
        "total_lines": len(lines),
        "changed_lines": 0
    }

    for i, line in enumerate(lines):
        if line.strip() != "expected_line":
            diff_result["differences"].append({
                "line_number": i + 1,
                "original": line.strip(),
                "expected": "expected_line"
            })
            diff_result["changed_lines"] += 1

    with open(output_file, 'w') as f:
        json.dump(diff_result, f, indent=2)

    if verbose:
        print(f"Found {diff_result['changed_lines']} changed lines out of {diff_result['total_lines']}")

This snippet shows how the tool reads input, processes each line, and writes the results to a JSON file. The actual implementation uses a more sophisticated diff algorithm, but the structure is similar.

One of the key benefits of using a self-contained CLI tool is that it avoids the need for external services or GUIs. This makes it ideal for integration into scripts or CI pipelines. For example, you can use it in a GitHub Actions workflow to validate changes in a config file before merging a pull request.

name: Validate Config Changes
on: [push]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Install TextDiffTool
        run: pip install textdifftool
      - name: Run Diff Check
        run: textdifftool --input config.json --output diff.json --verbose
      - name: Fail if changes found
        run: |
          if [ "$(cat diff.json | jq '.changed_lines')" -gt 0 ]; then
            echo "Changes detected in config file!"
            exit 1
          fi

This workflow demonstrates how you can integrate TextDiffTool into a CI/CD pipeline to ensure that no unintended changes are made to a config file.

For developers who need a reliable, lightweight way to compare text files line by line, TextDiffTool is an excellent choice. It provides a self-contained, command-line solution that can be embedded in scripts or used as a standalone utility. You can find the tool at https://intellitools.gumroad.com/l/textdifftool, where you'll also find a detailed README and requirements.txt to get started.