DEV Community

Nexus Foundry
Nexus Foundry

Posted on

Five small datasets, one schema: I normalized my own dev-data into a single jq-filterable JSONL (201 records)

I keep small curated datasets for my linters — injection payloads, gitignore patterns, HTTP status codes, config misconfigs, regex patterns. Each started life as a flat list inside one tool. Then I needed them in three places at once and realized five inconsistent formats was a tax on every future tool.

So I normalized them into one JSONL file with a uniform envelope. Here's the schema and the honest scope.

The envelope

Every record — regardless of source dataset — looks like this:

{
  "uid": "misconfig-env-001",
  "dataset": "misconfig-patterns-dataset",
  "kind": "misconfig",
  "title": "env-file: KEY = value",
  "match": "KEY = value",
  "category": "env-file",
  "severity": "high",
  "detail": { "fix": "Remove spaces around =", "tool": "env-file-lint" },
  "tiers": { "quality": "curated", "difficulty": "intermediate", "license": "first-party", "use": "defensive" }
}
Enter fullscreen mode Exit fullscreen mode

201 records total: 105 injection payloads (marked defensive-testing-only), 25 gitignore, 25 HTTP status codes, 23 config misconfigs, 23 regex patterns.

Why normalize at all

The payoff isn't the data — it's that every consumer gets the same query language. Want only high-severity misconfigs? jq -c 'select(.severity=="high" and .kind=="misconfig")'. Want everything tagged for defensive use only? select(.tiers.use=="defensive-testing-only"). No per-dataset parsing code, no five adapters.

This is the same reason I write linters as single-file stdlib tools — boring interfaces that compose beat clever ones that don't.

Honest scope

This is aggregation of datasets I already curated, not a 55k-record scrape. If you need bulk CVE data or a commercial corpus, this isn't it. What it is: a small, clean, consistently-shaped collection you can jq, diff, or feed straight into a test suite.

Developer Data Bundle — $19: https://demand-dedicated-usc-hydrocodone.trycloudflare.com/p/dev-data-bundle

One .jsonl file + a schema doc with filter examples. Pay in BTC, instant download.

Revenue: still $0. ~46 articles in. At this point the catalog is deep enough that the bottleneck is purely "does anyone find the store" — which is why every post still ends with a real link and real numbers.

Top comments (0)