Hi everyone 👋
In my previous update, I talked about why single-file JSON comparison is not enough for real regression testing.
Today I want to share my actual development process while building the core feature: folder-based batch JSON comparison for my offline QA toolkit.
The real pain with single-file comparison
When doing API regression testing, testers rarely compare only one pair of JSON files.
A full regression cycle usually generates dozens or hundreds of API response JSON files. Manually picking and comparing files one by one is extremely repetitive and time-consuming.
Existing diff tools either:
- Lack dedicated batch folder comparison for QA scenarios
- Cannot ignore dynamic fields in batch mode
- Fail to export clean, ticket-ready evidence in bulk What I’m building right now I’m currently developing the batch JSON diff module with QA regression workflow in mind:
- Load two entire folders (old version vs new version API responses)
- Auto-match corresponding JSON files by filename
- Skip predefined noisy fields (timestamp, traceId, requestId) in batch tasks
- Batch-generate standardized HTML comparison reports Key technical challenges I’m solving
- Batch field filtering consistency Single-file filtering is simple, but batch mode requires global rule persistence. I optimized the ignore-key configuration to take effect on all batch comparison tasks, ensuring unified filtering standards for the whole regression suite.
- Missing & new file detection During version iteration, some API files will be added or removed. The tool will mark new-added / deleted files separately in the final report, which is critical for full regression tracking.
- Batch HTML report generation Instead of generating separate messy files, I’m designing a structured batch report overview, so testers can quickly locate which API cases have differences. Current progress & next step Core folder traversal and file matching logic is nearly finished. I will complete the full batch comparison and batch export functions in the next few days. Once this feature is shipped, the toolkit will officially support full-scale API batch regression testing — the most demanded feature from enterprise QA teams. Quick question for QA testers What’s your biggest headache during batch API regression? Too many noise changes? Disorganized evidence? Feel free to drop a comment.
Follow me for more build-in-public progress of my offline QA desktop toolkit.
Top comments (0)