DEV Community

SEN LLC
SEN LLC

Posted on

How to Make Sense of a HAR File Without DevTools

How to Make Sense of a HAR File Without DevTools

A 2-second CLI summary for HTTP Archive files. Stdlib-only Python — json and argparse, that's the whole dependency list.

A coworker pastes a 4 MB .har file in Slack and writes "the product page is slow on staging, can you take a look?" You open it, see 60,000 lines of JSON, and your enthusiasm drops several rungs. You could load it into Chrome DevTools' Network panel — but that's a process, and it's per-browser, and your coworker already did that to make the file. WebPageTest has nice reports but you have to actually run the test there. What's missing is a 2-second local summary so the second person looking at the HAR can instantly see "ok, the hero image is 800 KB and there's no gzip on the CSS."

That's har-analyze. About 700 lines of Python, zero dependencies.

📦 GitHub: https://github.com/sen-ltd/har-analyze

Screenshot

docker run --rm -v "$PWD:/work" har-analyze capture.har --opportunities
Enter fullscreen mode Exit fullscreen mode

Problem: HAR is the lingua franca that nobody reads

HAR (HTTP Archive) is the default export format for browser DevTools Network panels — Chrome, Firefox, Safari, Edge, all of them. It's structurally simple JSON: log.entries[] is the list of requests, each entry has a request object, a response object, and a timings object. Teams paste HARs into Slack threads when debugging customer reports, ship them as bug-report attachments, and feed them into perf tools.

But the format is verbose to a fault. A 47-request page export is easily 80+ KB of JSON. There are fields nobody uses (request.cookies is always there, almost always uninteresting), redundant size fields (bodySize, headersSize, _transferSize), and timing semantics that subtly lie to you (startedDateTime is an ISO 8601 string but it is not monotonically increasing across entries, because parallel connections start before earlier ones finish). Reading one by hand to answer "what's the slowest request?" takes more time than running the page yourself.

Three tools already exist:

  • DevTools Network panel — but the panel is per-browser and per-machine; the person debugging from Slack doesn't have your tab.
  • WebPageTest — excellent, but you have to run a fresh test on their infrastructure. Doesn't apply to a HAR your coworker captured on staging behind their VPN.
  • jq over the HAR JSON — works, but you have to remember the schema and write the query, which is exactly what you're trying not to do.

What's missing: a CLI that takes a HAR and prints a useful summary in 2 seconds, that you can pipe | grep and that runs anywhere.

Design: which HAR fields actually matter

The whole tool is structured around the answer to "which HAR fields are signal, which are noise?" Here's the breakdown I landed on after a couple of hours staring at real HARs:

Field Useful?
request.url, request.method Yes, always
response.status Yes, for failures and redirects
response.content.size Yes — uncompressed body bytes, the number you actually want
response.content.mimeType Yes — used to bucket by type
response.headers Yes — content-encoding and cache-control drive every opportunity rule
time Yes — total request time in ms (browser-reported, not wall clock)
_transferSize (Chrome-only) Useful when present; fall back to bodySize + headersSize
startedDateTime ISO 8601 string, not monotonic — don't trust ordering
_initiator.type Best-effort; parser vs script powers the render-blocking heuristic
cache block Almost always empty in exported HARs. Skip it
request.cookies, response.cookies Always present, almost always uninteresting
pageTimings Inconsistent across browsers. Don't depend on it

The interesting headers — and the only ones the tool reads — are content-encoding, cache-control, and expires. Three headers cover four of the five opportunity rules.

Implementation

The whole thing is split into four pure modules and a thin CLI shell:

  • parser.py — HAR JSON → flat list of Entry records
  • analyzer.pyEntrys → Analysis (counts, top-N, failures, duplicates)
  • opportunities.pyEntrys → Opportunity list (rules)
  • formatters.pyAnalysishuman / json / markdown / csv
  • cli.pyargparse and exit codes

Pure functions all the way down. The only side effects are the file read in parse_har_file and the print at the end of cli.main. Tests build a synthetic HAR in conftest.py, so the test suite runs in 60ms and there's no fixture file checked in.

Snippet 1: normalizing entries

The parser's job is to take a HAR entry — which has nested request, response, content, timings objects with browser-specific quirks — and emit a flat Entry dataclass that the rest of the code can use without defensive coding.

@dataclass(frozen=True)
class Entry:
    url: str
    method: str
    status: int
    mime_type: str
    content_type: str  # bucketed: html, css, js, img, font, xhr, other
    domain: str
    size: int          # response body uncompressed size, in bytes
    transfer_size: int # bytes actually transferred (may be smaller if gzipped)
    time_ms: float     # total time for this request, in ms
    response_headers: dict[str, str]  # lowercased names
    request_headers: dict[str, str]
    initiator_type: str
Enter fullscreen mode Exit fullscreen mode

Two things that look small but matter a lot:

  1. Lowercase the header names. HTTP headers are case-insensitive but a HAR is a JSON dump, so you can get Content-Encoding, content-encoding, or even CONTENT-ENCODING. Lowercasing once at the parser boundary means every downstream rule can do entry.response_headers.get("content-encoding") without paranoia.
  2. Bucket the mime type at parse time, not at rule time. The full mime type is text/html; charset=utf-8 or application/javascript or application/vnd.api+json, and you don't want each rule to redo that string-matching dance. Map them once into seven buckets — html, css, js, img, font, xhr, other — and the rules become trivial.
_TYPE_BUCKETS: list[tuple[str, str]] = [
    ("text/html", "html"),
    ("text/css", "css"),
    ("javascript", "js"),
    ("image/", "img"),
    ("font/", "font"),
    ("application/json", "xhr"),
    # ... more
]

def _bucket(mime: str) -> str:
    m = (mime or "").lower().split(";", 1)[0].strip()
    for needle, bucket in _TYPE_BUCKETS:
        if needle in m:
            return bucket
    return "other"
Enter fullscreen mode Exit fullscreen mode

The split(";", 1)[0] strips the ; charset=... suffix that text mime types carry. First-match-wins ordering is deliberate — application/javascript should hit js before it could possibly hit application/json (it can't, but defensive ordering costs nothing).

There's also a small piece of wisdom hidden in the size handling:

size = int(content.get("size", 0))
if size < 0:
    size = 0  # HAR sometimes uses -1 for "unknown"
Enter fullscreen mode Exit fullscreen mode

HAR uses -1 as a sentinel for "we don't know this number" (cached responses, especially). Without that clamp, summing sizes silently goes negative and the "Total size" line in the summary becomes nonsense. The first version of the tool didn't handle this and showed -2.3 KB on a HAR with three cached entries, which is a great way to lose user trust on the first run.

Snippet 2: the missing-compression rule

This is the single most useful rule. Compressible text responses (HTML, CSS, JS, JSON) without a content-encoding header are leaving 70-80% on the table.

_COMPRESSIBLE_BUCKETS = {"html", "css", "js", "xhr"}

def _has_compression(entry: Entry) -> bool:
    enc = entry.response_headers.get("content-encoding", "").lower()
    return any(token in enc for token in ("gzip", "br", "deflate", "zstd"))

def check_missing_compression(entry: Entry) -> Opportunity | None:
    if entry.content_type not in _COMPRESSIBLE_BUCKETS:
        return None
    if entry.size < 1024:
        return None  # tiny responses don't matter
    if _has_compression(entry):
        return None
    if entry.status == 0 or entry.status >= 400:
        return None
    return Opportunity(
        rule="missing-compression",
        severity="warn",
        message=f"{entry.content_type} response served without compression "
                f"({entry.size} bytes) — add gzip or brotli at the edge",
        url=entry.url,
    )
Enter fullscreen mode Exit fullscreen mode

Five guard clauses, one return. Each guard has a real reason:

  • Bucket check: don't suggest gzipping a JPEG. JPEGs are already compressed; running gzip over them makes them slightly bigger.
  • < 1024 bytes: the gzip header itself is ~20 bytes, and below 1 KB the savings are negligible while the noise (every favicon flagged) is huge.
  • _has_compression: substring check, not equality. content-encoding: gzip and content-encoding: gzip, identity (legal, occasionally seen) both pass.
  • Status check: 4xx/5xx responses often come from error handlers that don't run through the normal compression middleware. Flagging them is noise.

Conservativism is the whole game here. A linter that cries wolf gets --ignored in three days and uninstalled in a week.

Snippet 3: tree-style human formatter

The default output is meant to look at home in a terminal:

def format_human(analysis, opportunities=None, *, color=False):
    out = []
    out.append(f"HAR summary: {analysis.page_url} "
               f"({analysis.request_count} requests, "
               f"{_fmt_bytes(analysis.total_size_bytes)}, "
               f"{_fmt_ms(analysis.total_time_ms)} total)")

    type_parts = [f"{b} {c}" for b, c in analysis.by_type.items()]
    out.append(f"├── By type:     {''.join(type_parts)}")
    # ... etc
    out.append(f"└── Duplicates:  ...")
    return "\n".join(out) + "\n"
Enter fullscreen mode Exit fullscreen mode

The box-drawing characters (├── / / └──) are how tree(1) and git log --graph render hierarchy and they reuse a piece of visual vocabulary every CLI user already knows. ANSI color is opt-in via a color: bool keyword and the codes are inlined directly — \033[32m for green, \033[0m for reset — because the alternative is pulling in colorama or rich and the whole point of this tool is "no dependencies." Adding a dep to color a few digits would be embarrassing.

The _fmt_bytes and _fmt_ms helpers do the human-readable conversion (1024 → "1 KB", 1500 → "1.5s"). They're 5 lines each.

Tradeoffs

Things this tool does not try to do, and why:

  • Wall-clock timing. The time field in HAR is browser-reported, not measured externally. If your DevTools was throttled to "Fast 3G" the times reflect throttled times; if you used "Disable cache" the cache field is meaningless. The tool reports what's in the HAR, not what reality is.
  • HTTP/2 server push detection. HAR doesn't expose pushed responses as a distinct concept — they look like normal requests with very low wait times. You can sometimes infer them from _initiator.type == "other" plus time < 5ms, but the false positive rate is awful and Chrome stopped supporting H2 push anyway. Skipped.
  • Connection reuse. HAR has serverIPAddress and connection fields, but they're inconsistent across browsers and rarely actionable. A 60-request page with bad connection reuse will jump out from the slowest section anyway.
  • Cache-aware analysis. A HAR captured on a hard reload looks very different from one captured on a soft reload — the latter shows lots of from cache entries with time: 0. The tool reports both honestly but doesn't try to reconstruct what a "cold" or "warm" load would have looked like. That's a much bigger project (basically reimplementing Lighthouse) and out of scope for a 2-second CLI.

The intent is to be the "first 30 seconds of looking at a HAR" tool. It's not Lighthouse, it's not WebPageTest, it doesn't try to be. It's the thing you run before deciding whether to bother running those.

Try in 30 seconds

git clone https://github.com/sen-ltd/har-analyze
cd har-analyze
docker build -t har-analyze .

# Capture a HAR from your browser DevTools, then:
docker run --rm -v "$PWD:/work" har-analyze capture.har

# With opportunities:
docker run --rm -v "$PWD:/work" har-analyze capture.har --opportunities

# JSON for scripting:
docker run --rm -v "$PWD:/work" har-analyze capture.har --format json | jq '.summary'

# Markdown for a PR comment:
docker run --rm -v "$PWD:/work" har-analyze capture.har --format markdown --opportunities

# Run the test suite:
docker run --rm --entrypoint pytest har-analyze -q
# 49 passed in 0.05s
Enter fullscreen mode Exit fullscreen mode

The Docker image is 60 MB total — Python 3.12 alpine plus exactly the project source. Zero pip dependencies means there's nothing to download except Python itself.

Closing

Entry #115 in a 100+ portfolio series by SEN LLC. Recent stdlib-only entries:

  • csvdiff — semantic CSV diff, key-based row matching
  • robots-lint — RFC 9309 robots.txt linter

If you regularly debug perf complaints from "real" production HARs, I'd love to know which other rules belong in the opportunity checks. The four current rules cover the cases I see most often, but there's a long tail of "X but only when Y" that I'd consider adding if it's a real recurring frustration.

Feedback welcome.

Top comments (0)