DEV Community: Deepak Kumar

How to Handle Large JSON Files Without Crashing Your Browser

Deepak Kumar — Thu, 30 Jul 2026 08:42:35 +0000

Files over 10 MB will slow most browser tools; over 50 MB will crash them. Here's how to handle large JSON without losing your mind.

JSON.parse() loads the entire file into memory before returning anything. For small files this is fine. For a 100MB database export, it will exhaust browser memory and crash the tab. This covers the practical strategies that actually work when JSON is too large to handle with standard tools.

The Problem: In-Memory Parsing

Every standard JSON parser — in browsers, Node.js, Python, and Java — works by reading the entire input, building a complete in-memory representation, and then returning it. A 100MB JSON file can expand to 500MB–1GB in memory, because the in-memory object graph has overhead for each key, value, and reference. Browser tabs typically have 512MB–2GB of memory limits and no swap. The result is a hard crash.

Rule of thumb:

Under 5MB: any browser formatter works fine
5–20MB: browser tools are slow but may work; use the strategies below for speed
Over 20MB: use command-line tools or streaming parsers

Strategy 1: jq — The JSON Swiss Army Knife

jq is a lightweight command-line JSON processor. It streams through the file without loading everything into memory and lets you filter, transform, and extract data.

# Pretty-print the first 10 array elements
jq '.[0:10]' large-data.json

# Get all keys of the root object
jq 'keys' large-data.json

# Count array items
jq 'length' large-data.json

# Extract a specific field from every object in an array
jq '.[].email' users.json

# Filter: only objects where age > 30
jq '[.[] | select(.age > 30)]' users.json

# Get only specific fields from each object
jq '[.[] | {name, email}]' users.json

# Convert array of objects to CSV
jq -r '.[] | [.name, .email, .age | tostring] | join(",")' users.json

Extract just the part you need, then use a browser formatter for the rest:

jq '.config.database' app-config.json > database-config.json
# Now paste database-config.json into a browser formatter

Strategy 2: Streaming Parsers (Node.js)

The stream-json npm package streams JSON token by token, so memory usage stays constant regardless of file size:

const { createReadStream } = require('fs');
const { chain } = require('stream-chain');
const { parser } = require('stream-json');
const { streamArray } = require('stream-json/streamers/StreamArray');

let count = 0;
chain([
  createReadStream('large-users.json'),
  parser(),
  streamArray()
]).on('data', ({ value }) => {
  // Process one object at a time — constant memory
  console.log(value.email);
  count++;
}).on('end', () => {
  console.log(`Processed ${count} records`);
});

Strategy 3: Streaming Parsers (Python)

ijson is the Python equivalent — it iterates over items in a large JSON array without loading the whole file:

import ijson

with open('large-users.json', 'rb') as f:
    for user in ijson.items(f, 'item'):
        # 'item' = each element of the top-level array
        print(user['email'])
        # Process one record at a time

For deeply nested paths:

import ijson

with open('response.json', 'rb') as f:
    # Extract items from data.users array
    for user in ijson.items(f, 'data.users.item'):
        process(user)

Strategy 4: Split the File into Chunks

If you need to work with all the data in a browser tool, split the large array into smaller files first.

# Using jq — split a 100,000-item array into chunks of 1,000
jq -c '.[]' large.json | split -l 1000 - chunk_
for f in chunk_*; do
  jq -s '.' "$f" > "${f}.json"
done

# Using Python
import json

CHUNK_SIZE = 1000

with open('large.json') as f:
    data = json.load(f)  # Still loads all into memory once

for i in range(0, len(data), CHUNK_SIZE):
    chunk = data[i:i + CHUNK_SIZE]
    with open(f'chunk_{i // CHUNK_SIZE}.json', 'w') as out:
        json.dump(chunk, out, indent=2)

Strategy 5: Import into a Database

For very large files (hundreds of MB or more) that you need to query repeatedly, importing into a database is the right long-term approach.

-- SQLite with JSON support
CREATE TABLE users AS
SELECT
  json_extract(value, '$.id') AS id,
  json_extract(value, '$.name') AS name,
  json_extract(value, '$.email') AS email
FROM json_each(readfile('users.json'));

-- PostgreSQL JSONB
COPY staging(raw) FROM '/path/to/data.json';
SELECT raw->>'name', raw->>'email'
FROM staging
WHERE (raw->>'age')::int > 30;

Strategy 6: JSONL (Newline-Delimited JSON)

If you control the data source, use JSONL format for large datasets. Each line is a valid, standalone JSON object — enabling line-by-line processing with standard Unix tools, no special parser needed.

{"id":1,"name":"Alice","email":"alice@example.com"}
{"id":2,"name":"Bob","email":"bob@example.com"}

wc -l data.jsonl                # Count lines (= record count)
jq -r '.email' data.jsonl       # Extract a field from every line
jq 'select(.age > 30)' data.jsonl  # Filter

with open('data.jsonl') as f:
    for line in f:
        record = json.loads(line.strip())
        process(record)

Summary: Which Strategy to Use

File Size	Best Approach
Under 5MB	Browser formatter
5–50MB	`jq` to extract the section you need, then browser tool
50MB–1GB	Streaming parser (`stream-json` / `ijson`) or `jq`
Over 1GB	Database import (SQLite or PostgreSQL)
Any size, repeated queries	Convert to JSONL or import to database

Compressing Large JSON for Storage and Transfer

A 100MB JSON file often compresses to 10–20MB with gzip, because JSON is highly repetitive text (repeated key names, whitespace, common values). Compression is almost always worth enabling for large JSON transmitted over HTTP.

gzip -k large-data.json          # creates large-data.json.gz, keeps original
gzip -d large-data.json.gz       # decompress
jq '.[]' large-data.json | gzip > extracted.jsonl.gz

// Node.js — stream with compression
const { createReadStream, createWriteStream } = require('fs');
const { createGzip } = require('zlib');

createReadStream('data.json')
  .pipe(createGzip())
  .pipe(createWriteStream('data.json.gz'))
  .on('finish', () => console.log('Compressed'));

For HTTP APIs, enable gzip on the server and set Accept-Encoding: gzip on the client. Most frameworks (Express, FastAPI, Spring Boot) have a one-line setting for this.

Diagnosing Memory Problems

Check the file size first — ls -lh data.json. Anything over 10MB shouldn't go into a browser formatter.
Count records without loading — jq 'length' data.json, or wc -l data.jsonl for JSONL. Know what you're dealing with before committing to loading everything.
Sample first — jq '.[0]' data.json to check structure before processing everything.
Profile Node.js memory — node --max-old-space-size=4096 script.js. If it still crashes, you need a streaming approach.
Python's memory_profiler — pip install memory-profiler, then mprof run script.py for per-line memory usage.

Large JSON in Cloud Storage

When large JSON files live in cloud storage, you have options that avoid downloading the whole file:

S3 Select — Amazon S3 can run SQL-like queries directly on JSON/CSV files in S3, returning only matching rows. You pay only for data scanned, far cheaper than downloading a 1GB file to extract 1,000 rows.
BigQuery external tables — query JSONL files in Cloud Storage directly, without importing them.
Azure Blob with Apache Spark — for multi-GB files, PySpark distributes parsing across a cluster.
Convert to Parquet — for repeated queries, Parquet reduces file size 5–10x and makes column-oriented queries 10–100x faster.

# Convert JSONL to Parquet with pandas
import pandas as pd

df = pd.read_json('large-data.jsonl', lines=True)
df.to_parquet('data.parquet', compression='snappy')

# Later: read back only the columns you need
df2 = pd.read_parquet('data.parquet', columns=['name', 'email', 'age'])

If you're under the 5MB threshold, JSON Formatter Hub handles formatting, validation, and tree/table views entirely in your browser — no upload required.

Why I Built a JSON Formatter That Never Sends Your Data Anywhere

Deepak Kumar — Thu, 23 Jul 2026 17:32:52 +0000

Every JSON formatter online does the same basic job: parse, validate, pretty-print. But most of them make the same architectural choice I decided not to make — they send your JSON to a server first.

I'm Deepak Kumar, a full-stack developer, and I built JSON Formatter Hub specifically to avoid that. Here's why it matters, and how the client-side version actually works.

The problem with server-side formatting

Think about what people actually paste into a JSON formatter: API responses, auth tokens, database exports, config files, sometimes entire request payloads with PII in them. A tool that uploads that to a backend — even briefly, even without storing it — is a tool you have to trust with data you probably shouldn't be sharing with a third party at all.

This isn't hypothetical. Newer "AI-powered" JSON tools go a step further: they pipe your JSON to an LLM API to generate explanations or fix errors. That means your data now also passes through a third-party AI provider's infrastructure. For anything containing credentials or user data, that's a meaningfully larger exposure surface than people realize.

Doing it entirely in the browser

The alternative is straightforward in principle, less so in practice: do everything with JSON.parse, JSON.stringify, and vanilla DOM rendering, entirely client-side.

A few specifics from building it:

Validation with useful error location. JSON.parse throws on invalid input, but the error message alone isn't enough for a developer to fix a 2000-line payload. I wrap it and back-calculate line/column from the character position in the error:

function validateJSON(input) {
  try {
    JSON.parse(input);
    return { valid: true };
  } catch (err) {
    const match = /position (\d+)/.exec(err.message);
    let line = null, column = null;
    if (match) {
      const pos = parseInt(match[1], 10);
      const lines = input.slice(0, pos).split('\n');
      line = lines.length;
      column = lines[lines.length - 1].length + 1;
    }
    return { valid: false, message: err.message, line, column };
  }
}

Tree and table views without a parsing library. Since the input is already a JS object after JSON.parse, tree view is just recursive DOM generation over that object — no separate parser needed. Table view is a flatten step: collect the union of keys across array entries, then map each row against that column set.

Large files. Pretty-printing a multi-megabyte JSON file synchronously can freeze the tab. Deferring the heavy formatting work and rendering progressively keeps the UI responsive without needing a server round-trip to do the same job slower.

The full implementation is open on GitHub: jsonformatterhub/jsonformatterhub.

Try it

JSON Formatter Hub is free, has no login, and — because of everything above — never sees your data. Format, validate, beautify, minify, or convert JSON to XML/CSV/YAML, all in your browser.

If you build developer tools yourself, I'd be curious whether you've made the same client-side-first call, or where it hasn't been worth it for you.

Online JSON Formatter and Validator for Developers

Deepak Kumar — Wed, 17 Dec 2025 17:55:32 +0000

JSON is widely used in APIs and web applications, but unformatted JSON can be hard to read and debug.

Free Online JSON Formatter and Validator

I built JSON Formatter Hub to help developers format, validate, beautify, and minify JSON quickly.

🔗 Tool link: https://jsonformatterhub.com/

What you can do with JSON Formatter Hub

Format JSON instantly
Validate JSON structure
Beautify or minify JSON
Convert JSON to XML, CSV, or YAML

Why use it?

No signup required
Runs fully in the browser
Fast and simple UI
Free for everyone

If you work with APIs or JSON data regularly, this tool can save you time.

👉 Try it here: https://jsonformatterhub.com/