DEV Community: m-naoki-m

jq, json.tool, or a browser formatter? How I pretty-print curl JSON

m-naoki-m — Thu, 09 Jul 2026 05:21:59 +0000

The actual problem

Every developer I know runs the same loop several times a day:

curl https://api.example.com/some-endpoint

The response comes back as one minified line of JSON. You squint, scroll, give up, and reach for a formatter.

I have ended up using three different formatters for this depending on the situation. None of these are novel — the interesting part is the picking. Below is the boring rule of thumb I actually follow, and the redaction step most posts about this skip entirely.

Approach 1: pipe through `jq`

This is the default and almost always the right answer:

curl -s https://api.example.com/users | jq .

✅ Fastest path to pretty output
✅ Tells you where the parse fails with a column reference
✅ Composable with filtering: jq '.users[] | select(.active)'
❌ Requires jq to be installed (not guaranteed on minimal containers or fresh CI)
❌ The filter mini-language has a learning curve

When to not reach for jq first: when you only need to read the response and you are already about to paste it somewhere else (a Slack thread, a GitHub issue, a ticket). At that point you are leaving the terminal anyway and the savings are marginal.

Approach 2: Python `json.tool` as a fallback

When jq is inconvenient to install — locked-down CI runners, remote SSH on a server you do not own, minimal Alpine images where you happen to have Python but not jq — the stdlib has you covered:

curl -s https://api.example.com/users | python3 -m json.tool

✅ Often available where installing jq is inconvenient
✅ Sufficient for pretty printing + parse validation
❌ Error messages are less precise than jq on truncated input
❌ No filter / select capability — it formats, that's it

I use this maybe twice a week. It is the "good enough" tool, not the "right" tool. Note that Python is not actually guaranteed either — distroless images and BusyBox containers ship with neither.

Approach 3: a browser-side formatter for the share step

I only paste curl output into a browser formatter when I am about to share the result with a teammate (Slack, GitHub issue, ticket review), and the payload is safe for that audience.

The reason that condition matters: a security firm disclosed last year that two popular online formatters had been retaining 80,000+ saved snippets, including AWS keys and JWTs. I covered the story separately in What the JSON Formatter Leaks Taught Me About Browser-Side Tools; here I only care about the practical rule it implies. Pick a static-site formatter that does no server-side conversion, verify in DevTools that no outgoing request carries your payload, and accept that this is something you re-verify per session and per build — not a permanent brand promise.

For my own sharing slot I use FormatArc's JSON formatter. It is a static site with no /api/* routes, the parser core is open source (repo), and the no-upload claim is checkable in your own DevTools before you paste anything real.

Redact before you share

This is the step most "format your curl response" posts leave out. Even when the payload looks innocent, it likely contains things that should not survive into a public Slack channel or a GitHub issue: API endpoints, internal IDs, error stack traces with file paths, customer references.

The pragmatic approach is to strip known secret-bearing keys with jq before you paste anywhere:

# Drop the obvious secrets at the top level
curl -s https://api.example.com/users \
  | jq 'del(.token, .password, .authorization)'

# Walk the entire tree and drop them at every depth
curl -s https://api.example.com/users \
  | jq 'walk(if type == "object"
             then del(.password, .token, .secret, .authorization)
             else . end)'

The second form catches secrets nested inside response bodies (data[].credentials.token, result.session.authorization, etc). I have it as a shell alias because I want zero friction between "I should redact this" and actually doing it.

A picking heuristic

Situation	Use
Local terminal, exploring response shape	`jq`
Local terminal, need to filter or transform	`jq` (or `mlr` if the data is tabular)
CI / Docker without `jq` (but with Python)	`python3 -m json.tool`
Pasting to share — already redacted, public-safe	browser formatter
Pasting to share — payload contains anything internal	redact in `jq` first
Large file (> 100 MB)	`jq --stream` or DuckDB, not a browser

For most of these rows, jq is the answer. The "switching between three" framing is mostly an excuse to write down the boundaries — that is where I see people lose time.

What I am explicitly not claiming

jq is not "my discovery." It has been the default in this niche for over a decade. python3 -m json.tool is well-known. The only thing I am asserting here is that the boundaries between these three approaches, plus the redaction step, are where the friction actually lives.

A few honest limits worth flagging:

A browser-side formatter is not immune to leaks — the threat model still includes XSS in the parser, malicious extensions reading the DOM, supply-chain hits on npm deps. The browser is a smaller blast radius than a server-side formatter, not a zero one. Re-verify the no-upload claim in DevTools per session or build; do not trust it as branding.
python3 -m json.tool is the fallback. If jq is reachable, prefer jq.

Try them

jq: https://stedolan.github.io/jq/
FormatArc JSON formatter: https://formatarc.com/en/json-formatter/

For a deeper dive on why JSON specs disallow trailing commas or comments (with cross-parser verification against GitHub, marked, and remark-gfm), see Are JSON comments allowed? RFC 8259 and 4 fixes. It is the article Google's AI Overview cites when asked about ECMA-404 / RFC 8259 comment rules.

If you have a redaction recipe better than the walk(...) filter above — especially for nested envelopes from common API frameworks (Laravel, Rails, NestJS, Spring) — I would like to read it.

Building an Orthodontic Progress App That Never Uploads Your Face

m-naoki-m — Sat, 04 Jul 2026 14:09:19 +0000

I recently shipped SmileProgress (Japanese name: 矯正ログ / Kyousei Log), an indie iOS app for people going through orthodontic treatment — braces, aligners, and everything in between. The core promise is unusual: you can track months or years of progress without ever putting a photo of your face on a server. Not mine. Not iCloud. Not anyone's.

This post is a technical write-up of the on-device anonymization pipeline that makes that promise possible. It uses only Apple's built-in frameworks — Vision, ImageIO, CoreImage — and stores everything in the user's local Documents directory.

If you're building anything privacy-sensitive on iOS and you want a working example that goes beyond "we hash your email," I hope this is useful.

Why the constraint matters

Orthodontic treatment takes 1–3 years. People want to see the change, but they almost never want to publish "open your mouth wide" photos of their own face to a cloud service. In the Japanese 矯正垢 (orthodontic accounts) community on Instagram and X, users routinely crop, blur, or paint over everything except the teeth before sharing.

So the design constraint was:

The app records mouth photos for years, but never uploads the raw photo
Only the mouth region — cropped, stripped of metadata — can be shared, and only when the user explicitly opts in
No account. No sync server. No cloud storage bill I have to pay every month as an indie developer

All three requirements point at the same technical answer: do the anonymization on the device, right after capture.

The pipeline in one picture

The full flow inside the phone looks like this:

AVCaptureSession grabs a frame from the camera
VNDetectFaceLandmarksRequest finds the face and its landmarks
outerLips.normalizedPoints gives us the mouth contour
We compute a square crop with a 1.6× margin
CGImageDestination writes a fresh JPEG with no EXIF / GPS / TIFF metadata
PhotoStore.add(...) saves it into Documents/photos/ with a JSON sidecar

Nothing in that list touches the network.

Detecting only the mouth

VNDetectFaceLandmarksRequest returns all facial landmarks — eyes, nose, jaw, pupils, and (importantly for us) the outer lip contour.

import Vision

func detectMouth(in image: UIImage) async -> [CGPoint]? {
    guard let cgImage = image.cgImage else { return nil }

    return await withCheckedContinuation { continuation in
        let request = VNDetectFaceLandmarksRequest()
        request.revision = VNDetectFaceLandmarksRequestRevision3

        let handler = VNImageRequestHandler(
            cgImage: cgImage,
            orientation: image.cgImageOrientation
        )

        do {
            try handler.perform([request])
            guard let face = request.results?.first,
                  let outerLips = face.landmarks?.outerLips else {
                continuation.resume(returning: nil)
                return
            }
            continuation.resume(returning: outerLips.normalizedPoints)
        } catch {
            continuation.resume(returning: nil)
        }
    }
}

Two things are worth calling out in that snippet.

First, revision = VNDetectFaceLandmarksRequestRevision3 (available iOS 15+) is a huge accuracy jump over the older revisions. If you're serious about landmark work, don't rely on the default revision — pin the newest one that ships with your minimum target.

Second, outerLips.normalizedPoints is not in image coordinates. It's normalized to the detected face bounding box. Which brings us to the next surprise.

The coordinate system will trip you up

outerLips.normalizedPoints gives you (x, y) in the range 0..1, but that range is measured relative to the face bounding box. And Vision uses a bottom-left origin, while UIImage and CGImage use top-left. Convert both:

let faceBBox = face.boundingBox                 // normalized to full image, 0..1
let imgWidth = CGFloat(cgImage.width)
let imgHeight = CGFloat(cgImage.height)

func denormalize(_ p: CGPoint) -> CGPoint {
    let x = (faceBBox.origin.x + p.x * faceBBox.width) * imgWidth
    let y = (1.0 - (faceBBox.origin.y + p.y * faceBBox.height)) * imgHeight
    return CGPoint(x: x, y: y)
}

let lipPointsInImage = outerLips.normalizedPoints.map(denormalize)

The Y-axis flip is the classic Vision gotcha. Every couple of months I forget it and spend twenty minutes debugging a phantom rotation.

The 1.6× margin trick

Now we have the lip contour in image pixels. The naive move is: bounding-box the points and crop exactly that rectangle. That looks awful. A raw lip crop with no surrounding skin feels clinical and slightly uncanny.

I experimented with margins from 1.0× to 2.4× and settled on 1.6× — enough to include a strip of skin, chin, and philtrum, not so much that it starts leaking useful identifying features:

let xs = lipPointsInImage.map(\.x)
let ys = lipPointsInImage.map(\.y)
guard let minX = xs.min(), let maxX = xs.max(),
      let minY = ys.min(), let maxY = ys.max() else { return nil }

let lipBBox = CGRect(
    x: minX, y: minY,
    width: maxX - minX, height: maxY - minY
)

let center = CGPoint(x: lipBBox.midX, y: lipBBox.midY)
let side = lipBBox.width * 1.6
let squareRect = CGRect(
    x: center.x - side / 2,
    y: center.y - side / 2,
    width: side, height: side
)

Cropping to a square (not the lip aspect ratio) matters for a later feature — a Before/After slider that only works when frames share dimensions.

When the mouth is wide open, landmarks fly off

Real orthodontic photos are almost always taken with the mouth wide open to expose the teeth. Vision's landmark model was clearly trained on more neutral expressions, and when the jaw drops significantly, outerLips can return points that are 20–30% off the actual lip line.

The heuristic I ended up with is a corner-angle check: if the left and right corners of the lip contour form an angle that couldn't plausibly be a mouth on a roughly upright face, fall back to a different detector.

let leftCorner  = lipPointsInImage.min(by: { $0.x < $1.x })!
let rightCorner = lipPointsInImage.max(by: { $0.x < $1.x })!
let angle = atan2(
    rightCorner.y - leftCorner.y,
    rightCorner.x - leftCorner.x
)

if abs(angle) > .pi / 3 {
    return await fallbackDetection(cgImage)
}

The fallback path uses simple HSV color analysis — detecting the teeth (bright, slightly blue-shifted white) and the lip (red-shifted) directly in pixel space. It's lower quality than Vision, but it recovers cases where VNDetectFaceLandmarksRequest returns garbage or nothing at all.

Close-up shots have no face at all

The other reality check: users import old photos from their camera roll, and orthodontic patients tend to shoot super close — just the mouth, no face in frame. In that case VNDetectFaceLandmarksRequest returns zero results, and no amount of tweaking will fix it.

So the whole detector is a two-stage cascade:

static func detectMouth(in image: UIImage) async -> MouthLandmarks? {
    // 1. Vision (works when a full face is visible)
    if let landmarks = await detectViaVision(image) {
        return landmarks
    }
    // 2. Color-analysis fallback (works on close-up shots)
    return await detectViaColorAnalysis(image)
}

Stripping EXIF, GPS, and TIFF metadata

Anonymizing the pixels is only half the job. iPhone photos ship with EXIF (timestamp, exposure), TIFF (device model, software version), and GPS (home address, to be blunt) blocks embedded in the file. If you UIImage.jpegData(compressionQuality:) and write that, all of it gets carried along.

The correct move is to skip UIImage entirely and use ImageIO's CGImageDestination, then simply omit every metadata dictionary:

import ImageIO
import MobileCoreServices

func writeAnonymizedJPEG(cgImage: CGImage, to url: URL) throws {
    guard let destination = CGImageDestinationCreateWithURL(
        url as CFURL,
        kUTTypeJPEG,
        1,
        nil
    ) else {
        throw AnonymizationError.destinationCreationFailed
    }

    let options: [CFString: Any] = [
        kCGImageDestinationLossyCompressionQuality: 0.92
        // deliberately no Exif / GPS / TIFF dictionaries
    ]

    CGImageDestinationAddImage(
        destination,
        cgImage,
        options as CFDictionary
    )

    guard CGImageDestinationFinalize(destination) else {
        throw AnonymizationError.finalizeFailed
    }
}

I also added a self-test that reads the JPEG back and asserts every metadata dictionary is nil. In DEBUG builds, this runs after every save. It's cheap paranoia and it caught a real regression once when a refactor accidentally started passing the source properties through.

func assertNoMetadata(url: URL) throws {
    guard let source = CGImageSourceCreateWithURL(url as CFURL, nil),
          let props = CGImageSourceCopyPropertiesAtIndex(source, 0, nil)
            as? [CFString: Any] else {
        throw AnonymizationError.readbackFailed
    }
    assert(props[kCGImagePropertyExifDictionary] == nil)
    assert(props[kCGImagePropertyGPSDictionary]  == nil)
    assert(props[kCGImagePropertyTIFFDictionary] == nil)
}

The full pipeline

Chained together:

static func anonymizeAndSave(
    image: UIImage,
    to url: URL
) async throws {
    // 1. Normalize orientation so pixel space matches display space
    let normalized = image.fixedOrientation()

    // 2. Detect the mouth (Vision, then color-analysis fallback)
    guard let landmarks = await detectMouth(in: normalized) else {
        throw AnonymizationError.mouthNotDetected
    }

    // 3. Compute a square with 1.6× margin around the lip bbox
    let square = squareRect(from: landmarks, margin: 1.6)

    // 4. Crop
    guard let cgImage = normalized.cgImage,
          let cropped = cgImage.cropping(to: square) else {
        throw AnonymizationError.cropFailed
    }

    // 5. Write a fresh JPEG with no EXIF / GPS / TIFF
    try writeAnonymizedJPEG(cgImage: cropped, to: url)

    // 6. Verify (DEBUG only)
    #if DEBUG
    try assertNoMetadata(url: url)
    #endif
}

Six steps, all local. There is nowhere in this pipeline where the raw photo — the one with the face still in it — leaves the phone.

Why on-device is the right default for an indie

I'll be honest: I initially considered a Firebase / Supabase-backed version because it would have been faster to prototype. Two things pushed me back to on-device only.

Cost predictability. Orthodontic photos are chunky (multi-megabyte JPEGs) and each user might store hundreds or thousands over a treatment. As an indie developer, "photo storage that scales with your active users" is exactly the kind of open-ended monthly bill I do not want. On-device storage means my hosting costs are effectively my smileprogress.com marketing site.

Trust as a product feature. In the orthodontic community, "does this app leak my face photos?" is a live question. Every legacy app in the space has had at least one review complaining about the cloud requirement. Building the entire app around "we cannot upload your raw photos even if we wanted to" turned out to be the strongest positioning I've had for any indie project so far.

The tradeoff — and it is a real one — is that features requiring backend state (multi-device sync, remote backup, shareable galleries) are all harder. I address the shareable-gallery case by making the sharing action extremely explicit: user taps Share, gets a preview of exactly what will be sent, has to accept a Terms + Privacy modal, and only then the already-anonymized mouth crop is uploaded. The face never leaves the device at any point in that path.

Try SmileProgress

The app is called SmileProgress (Japanese: 矯正ログ) and it's shipping on the App Store today, initially Japan-first and expanding to other regions:

App Store: https://apps.apple.com/app/id6780383136

It's a one-time purchase — currently a launch price of ¥160 in Japan (regional equivalent in other App Store regions) until 2026-12-31 — with no subscription, no ads, and no cloud-backed features hidden behind a future paywall. Once you own the app, the record / anonymize / compare / anonymous-share flow is yours to keep.

If you're building something in a privacy-adjacent space and want to compare notes on on-device pipelines, App Attest for anonymous auth, or output: "export" static SSG for a companion gallery site, I'd love to hear from you in the comments. Individual developer, so I read everything.

Recap

VNDetectFaceLandmarksRequest (revision 3) gives you outerLips.normalizedPoints, but they're normalized to the face bounding box and use a bottom-left origin
A 1.6× margin around the lip bounding box strikes a decent balance between "clinical crop" and "leaks features"
Wide-open mouths break Vision — cascade to a color-analysis fallback for close-ups
CGImageDestination with no metadata dictionaries is the reliable way to strip EXIF/GPS/TIFF; a DEBUG self-test catches regressions
Six steps, all on-device, zero network traffic for the raw photo

Thanks for reading. Happy shipping.

What the JSON Formatter Leaks Taught Me About Browser-Side Tools

m-naoki-m — Wed, 24 Jun 2026 01:54:14 +0000

A formatter leak that should change how you pick tools

In late November 2025, security firm watchTowr published a disclosure about two of the most popular online JSON formatter sites — jsonformatter.org and codebeautify.org. Both offered a "Recent Links" feature that let users share formatted output with a teammate. The URLs were predictable, which meant the saved content was effectively reachable to anyone who guessed the URL pattern.

Per the watchTowr report, the result was over 80,000 saved snippets, totaling 5 GB+ of data, sitting on the public internet without authentication. The content that researchers were able to retrieve included:

Active Directory credentials
Cloud access keys (AWS / Azure / GCP)
Private keys and CI/CD secrets
JWTs from production systems
AWS Secrets Manager exports

Based on the content of individual snippets, the data appeared to originate from a wide range of organizations — government, finance, healthcare, aerospace. Full disclosure: watchTowr — "Stop Putting Your Passwords Into Random Websites".

Two things worth flagging up front:

The vulnerable surface was the saved-and-shared feature, not every act of pasting into the formatter. Plenty of online formatters do not retain saved content, and some have already responded to the disclosure. It is not a blanket "all online tools are leaking" story.
The lesson generalizes anyway: if a tool can persist your input on its servers, your threat model has to assume it might, until proven otherwise.

How to actually verify what a formatter does

Most developers I know pick a formatter the same way they pick a coffee shop: whichever is closest, whichever rendered first. The leak suggests we should be slightly more deliberate. A few questions you can answer in under a minute per tool:

Open DevTools → Network, paste your input, hit format. Does any outgoing request carry a payload that contains your input or the converted output? If yes, your data left the browser. Hostnames matter — first-party analytics pings are different from a POST /api/format.
Is there a "save and share" / "recent links" / "history" feature? If yes, your input is probably being stored, even if the URL looks ephemeral.
Is the conversion logic open source? Not the marketing site — the actual parser. A static site that links to its own source repo is much easier to verify than a closed page.
Is there a CLI version with the same engine? If yes, you can usually verify the parser end-to-end with offline tools (strace, tcpdump, sandboxed Docker run) before trusting the web version.

This is a generic checklist. It is not specific to any tool, including mine.

What I built, and what I am not claiming

For my own use I built FormatArc — a converter for JSON, YAML, CSV, Markdown, and HTML. The architecture choices are not novel; they just happen to make some of the questions above answerable:

Next.js with output: "export" — pure static site, no server runtime
All parsing in the browser (JSON.parse, js-yaml, papaparse, marked, turndown, remark)
No /api/* routes in the project (you can grep app/ on the CLI repo for the same parser core)
No persistence of input to localStorage or IndexedDB by design — input lives in React state and goes away with the tab

What I will not claim:

"Zero network requests." Open DevTools and you will see chunk loads, fonts, the Next.js prefetch, and so on. The honest claim is narrower: no outgoing request whose payload contains the pasted input or the converted output. That is the one that matters for the leak threat model.
"Works offline." I have not shipped a Service Worker, so a real offline guarantee does not exist. The right phrasing is "no upload of your data," not "works without network."
"Bulletproof." Browser-side tools have their own failure modes — XSS in a parser, a compromised CDN, a malicious browser extension reading the DOM, a supply-chain hit on an npm dep. None of these are eliminated by removing the backend. The browser is a smaller blast radius, not zero.

A comparison that I think is actually verifiable

I tried to write this table in terms you can verify yourself rather than self-flattering bullet points:

	FormatArc	jsonformatter.org	codebeautify.org
Static site (no server-side conversion)	✅	❌	❌
Saved-and-share / history feature	❌	✅	✅
Conversion engine open source	✅ (repo)	❌	❌
Same engine available as a CLI	✅ (`npx formatarc`)	❌	❌
Multi-format (JSON + YAML + CSV + Markdown + HTML)	✅	partial	partial

If any cell is wrong by the time you read this, file an issue — I would rather correct it than carry a stale claim. The two competitor sites do change their feature sets.

When CLI tools are still the right answer

For shell pipelines, CI scripts, and anything bigger than a paste window, dedicated CLIs are usually still the right tool. jq, yq, csvkit, mlr, and pandoc cover the structured-query and large-file cases better than any browser tool. FormatArc is for the interactive, paste-and-go work where you specifically don't want the input ending up on someone else's server.

For the overlap case, you can use the same parser core from the CLI:

# One-shot convert
npx formatarc json-format '{"name":"FormatArc","tools":3}'

# Pipe-friendly
cat config.yaml | npx formatarc yaml-to-json > config.json

A note on npx: it is convenient but not a security guarantee on its own. Pin a version, audit the dependency tree, or install from source if you are running this inside a sensitive environment.

Try it, break it, tell me

Web: https://formatarc.com
Source: https://github.com/m-naoki-m/formatarc

If you find a case where the verification claims above do not hold — a request that leaks input, an edge case where the parser misbehaves, a YAML 1.1 vs 1.2 inconsistency — open an issue. I would rather hear it from you than learn it the hard way.