jq streams: when your JSON is too nested for regular paths

You know the problem. You've got a 500KB OpenAPI spec and need to strip every description field at any depth. Regular jq paths like .. | .description? work for reading, but try deleting them and you're stuck — the path doesn't tell you where to put the result back.

That's what tostream and fromstream are for.

tostream flattens the entire document into a stream of [path, value] pairs:

jq 'tostream' spec.json | head -5
# [["openapi"], "3.0.1"]
# [["info", "title"], "Petstore"]
# [["info", "description"], "This is a sample server..."]

Now you can filter with normal tools — grep -v, jq 'select(.[1] | type != "string" or . != "description")', whatever — then pipe to fromstream to reconstruct:

jq 'tostream | select(.[0][-1] != "description") | fromstream' spec.json > spec-clean.json

That one-liner drops every description key at any depth. The stream format is [path_array, leaf_value] — so .[0][-1] gives you the last key in the path.

Caveats:

• Loses key order (JSON objects are unordered anyway)
• Streams the whole doc in memory; fine for a few MB, not for GB
• fromstream expects valid stream format — if your filter breaks the structure, it errors

I used this last week on a 2MB Kubernetes CRD manifest to remove all x-kubernetes-* vendor extensions. Took 3 seconds. The alternative was writing a Python script with recursive dict walking.

jq's streaming filters have been there since 1.6 (2019). Not new, just underused. Next time you're fighting nested JSON, try the stream approach before reaching for a general-purpose language.

---

Reference: Lars Kellogg-Stedman's writeup on jq streams (blog.oddbit.com, 2023)