DEV Community

mi8bi
mi8bi

Posted on

Bulk-validating JSON test cases against a CSV schema with JSONPath (jsonpath-plus)

#json #typescript #testing #javascript

What this is

Of all the tools on Torinoa Tools, the one I personally use the most is the JSON Expected Value Validator. When working on APIs, I regularly need to check that a specific key in many different JSON test cases has the expected type or value. This tool lets you define validation rules with JSONPath in a CSV, then apply them across many JSON test cases at once.

What it does

  • Schema CSV (row-based): each row defines a JSONPath to check, the expected type, whether it's required, and a comparison mode
  • Test case JSON(s): upload or paste multiple JSON documents to validate
  • The tool cross-references both and shows a pass/fail grid — one row per rule, one column per test case

A schema CSV looks like this:

key,expected_type,required,compare_mode
$.user.id,number,true,
$.user.roles,array,true,unordered
$.user.email,string,false,
Enter fullscreen mode Exit fullscreen mode

Evaluating JSONPath with jsonpath-plus

Path evaluation is handled by jsonpath-plus:

import { JSONPath } from "jsonpath-plus";

matches = JSONPath({ path: normalizePath(rule.key), json: data, wrap: true });
Enter fullscreen mode Exit fullscreen mode

Passing wrap: true means zero, one, or many matches are all returned as an array, uniformly. That lets a path matching multiple elements ($.items[*].id) and a path matching exactly one ($.user.id) go through the same comparison logic downstream.

There's also light normalization so users can type user.id instead of $.user.id and still get a working path:

function normalizePath(rawKey: string): string {
  const key = rawKey.trim();
  if (key.startsWith("$")) return key;
  if (key.startsWith(".")) return "$" + key;
  return "$." + key;
}
Enter fullscreen mode Exit fullscreen mode

Comparison modes

Rather than only supporting exact equality, there are four comparison modes:

function compareValues(
  expected: unknown,
  actualList: unknown[],
  mode: CompareMode,
): { pass: boolean; message: string } {
  if (mode === "length_only") {
    const expectedLen = typeof expected === "number" ? expected : Number(expected);
    const pass = actualList.every((a) => structuralLength(a) === expectedLen);
    return { pass, message: pass ? t("msgValuePass") : t("msgValueFail") };
  }
  if (mode === "contains") {
    const pass = actualList.every((a) => containsMatch(expected, a));
    return { pass, message: pass ? t("msgValuePass") : t("msgValueFail") };
  }
  const arrayMode: "exact" | "unordered" = mode === "exact" ? "exact" : "unordered";
  const pass = actualList.every((a) => deepEqual(expected, a, arrayMode));
  return { pass, message: pass ? t("msgValuePass") : t("msgValueFail") };
}
Enter fullscreen mode Exit fullscreen mode
  • exact — full equality, array order included
  • unordered — arrays match as sets; order doesn't matter
  • contains — the expected value just needs to appear within the actual value (partial match)
  • length_only — only checks the length of an array or string, not its contents

When testing API responses, "the array's contents should match but order isn't guaranteed" comes up constantly, so unordered is the default comparison behavior. When a JSONPath matches multiple elements, the same comparison runs against every match, and every() requires all of them to pass for the rule to succeed.

The bug that took the longest to track down: CSV escaping

The hardest part of building this tool wasn't the JSONPath logic — it was CSV escaping. Schema JSONPaths sometimes contain filter expressions with embedded double quotes, e.g. $.items[?(@.type=="premium")]. Treated as a single CSV field, that needs proper RFC 4180 quoting, or the string silently corrupts on parse.

export function escapeCsvField(value: string, delim: string): string {
  if (
    value.includes(delim) ||
    value.includes('"') ||
    value.includes("\n") ||
    value.includes("\r")
  ) {
    return `"${value.replace(/"/g, '""')}"`;
  }
  return value;
}
Enter fullscreen mode Exit fullscreen mode

A field only gets wrapped in quotes if it contains the delimiter, a double quote, or a newline — and any embedded double quotes get doubled up. This got pulled out into a shared utility (src/utils/csvParse.ts) used by every CSV-related widget on the site. Before that fix existed, sample data with a double-quoted JSONPath filter would silently corrupt the CSV on parse — the kind of bug that's easy to miss because nothing throws, the data just quietly comes out wrong.

Parsing lives in the same file:

export function parseRow(row: string, delim: string): string[] {
  const fields: string[] = [];
  let cur = "";
  let inQ = false;
  for (let i = 0; i < row.length; i++) {
    const ch = row[i];
    if (ch === '"') {
      if (inQ && row[i + 1] === '"') {
        cur += '"';
        i++;
      } else {
        inQ = !inQ;
      }
    } else if (ch === delim && !inQ) {
      fields.push(cur);
      cur = "";
    } else {
      cur += ch;
    }
  }
  fields.push(cur);
  return fields;
}
Enter fullscreen mode Exit fullscreen mode

Delimiter detection is automatic too — it inspects the first line, ignoring anything inside quotes, and picks whichever of comma/tab/semicolon/pipe appears most — so pasting in CSV, TSV, or semicolon-separated data all just work without asking the user to specify a format up front.

Wrap-up

jsonpath-plus handles the actual path evaluation, but the real design work was elsewhere: how to treat single vs. multiple matches uniformly, how many comparison modes are actually worth supporting, and how strict to be about CSV escaping. That last one is easy to underestimate, but it directly affects whether the validation results you're looking at are actually trustworthy — worth building carefully as a shared utility rather than re-solving per widget.

Try it here: https://tools.torinoa.com/tools/json-expected-value-validator/

Top comments (0)