DEV Community

jerrywang
jerrywang

Posted on

Feature Finished: Batch JSON Diff for Massive API Response Regression Testing

Hi everyone 👋

Great milestone update for my offline QA desktop toolkit.

I’ve officially finished the full batch JSON comparison module — the most requested feature for real API regression testing workflows.

Earlier versions only supported single-file JSON diff. While helpful for quick checks, it could not handle real-world regression cycles where testers need to verify dozens or hundreds of API response files in one go.

What’s new in this release

The new batch JSON feature solves three major QA pain points:

1. Full folder-level comparison

You can now select two folders: one for old-version API responses and one for new-version responses. The tool automatically matches JSON files by filename and runs comparison in bulk.

2. Global ignore key rules (batch-wide filtering)

Dynamic noise fields like timestamp, traceId, requestId, and random tokens are the biggest annoyance in API regression.

I implemented global ignore configuration. Once configured, the rules apply to every single JSON file in the batch task. No more manually skipping noise fields for each file.

3. New / missing file detection

In version iterations, some API interfaces will be added, removed, or renamed.

The toolkit now clearly identifies:

  • Newly added JSON test cases
  • Deleted / deprecated test cases
  • Cases with actual business-level field changes

4. Consolidated batch HTML report

Instead of generating scattered files, the tool outputs one unified HTML report for the entire batch regression task.

Testers can:

  • View all comparison results in one page
  • Quickly locate failed / changed cases
  • Attach the complete evidence file directly to Jira tickets

Why this is a game-changer for QA

Most generic diff tools are built for developers, not testers.

They lack batch filtering, cannot flag missing test cases, and produce messy reports that are not ticket-friendly.

This toolkit is QA workflow-first:

  • 100% local offline processing
  • No test data uploaded anywhere
  • Auto-filter noise changes
  • Output clean, audit-ready regression evidence

Next roadmap

With batch JSON comparison fully completed, I will move to the next core module: batch PDF text comparison for document and report regression.

Stay tuned for the next update.


Follow my build-in-public journey for more offline QA tool updates.

BuildInPublic #QA #TestAutomation #API #RegressionTesting #Java #DesktopTools

Top comments (0)