DEV Community: Iulian Stoian

Writing a decoder that gives up on purpose

Iulian Stoian — Mon, 31 Aug 2026 15:48:04 +0000

I write decoders for WebSocket payloads. Someone's app sends 214 bytes, I have no schema, and I have to guess what they are.

The obvious way to build this is a chain of attempts: try JSON, try base64, try gzip, try MessagePack, return whatever sticks. That's what I built first. It was worse than useless, and fixing it changed how I think about this kind of code.

Wrong answers cost more than no answer

MessagePack is where I learned this. It's self-describing — the first byte tells you the type — so a decoder can start reading almost any byte sequence and produce something. Feed it a chunk of an encrypted blob and it will hand you back a map with two keys and an array, confidently.

You then spend twenty minutes trying to work out what that structure means in the app you're debugging. It means nothing. It's noise that happened to start with a valid type byte.

The fix is one rule: a decoder either consumes the entire payload or it fails. No leftover bytes, no partial match, no "close enough". If 200 of 214 bytes parse and 14 are left over, that's a rejection, not a result.

function decode(bytes) {
  const { value, consumed } = tryMessagePack(bytes)
  if (consumed !== bytes.length) throw new Error('partial')
  return value
}

Boring rule, big effect. False positives dropped to roughly zero, and I stopped debugging my own decoder's imagination.

There's a second-order version of the same problem. A Thrift Compact payload without a schema decodes fine structurally — you get field 1, field 2, field 3 with their types. But the field names live in the schema you don't have, so what you're looking at is f1: 8172, f2: "…". That's honest, and occasionally useful, and it's important the UI doesn't dress it up as more than it is.

Some things can't be decoded, and that's not a bug

Discord's gateway taught me this one.

It can compress with zlib-stream. Not "each message is zlib-compressed" — a single zlib stream, opened once when the connection opens, with every message appended to it. Each message ends with 00 00 FF FF, the Z_SYNC_FLUSH marker.

The consequence took me a while to accept: you cannot decompress message 400 on its own. zlib builds a dictionary from everything that came before. Message 400 is compressed against messages 1 through 399. Without them you don't have partial information, you have none.

So the decoder needs a persistent decompression context per connection, fed every frame in order, from the start. Mine keeps one DecompressionStream('deflate') alive per connection, with a queue so frames can't overtake each other, and correlates output back to input frames by the flush boundaries.

And if you attach in the middle of a conversation, it returns nothing. Not corrupted output — nothing. I spent a while looking for the bug before accepting there isn't one. The bytes genuinely don't contain the message.

That limitation is worth surfacing to the user rather than hiding, because it tells them something actionable: reload the page with the tool already running, and it works.

It's also the strongest argument I know for capturing from page load instead of from the moment someone opens DevTools. Stream compression makes late attachment useless, and the handshake is where the interesting failures live anyway.

Detecting what you can't read

If a decoder is going to give up honestly, it needs to know the difference between "I don't have a decoder for this" and "nobody has a decoder for this, it's encrypted".

The obvious measurement is entropy. Encrypted data should look random, compressed and encoded data less so. I implemented it and it doesn't work.

Measured on real payloads:

encrypted blob: 5.3 bits/char
base64-encoded JSON: 5.1 bits/char

Those distributions overlap. Pick a threshold anywhere in that range and you'll mislabel readable data as encrypted, or the reverse, depending on which way you lean. Base64 is just dense enough to look like ciphertext by that measure.

What works is dumber. Base64-decode the string, then ask what fraction of the resulting bytes are printable characters:

function looksOpaque(bytes) {
  let printable = 0
  for (const b of bytes) {
    if (b === 9 || b === 10 || b === 13 || (b >= 32 && b < 127)) printable++
  }
  return printable / bytes.length < 0.75
}

Under about 75% printable and it's encrypted or otherwise opaque. Decoded JSON, decoded text, decoded XML — all overwhelmingly printable. Ciphertext isn't. Two clean populations, no overlap in practice.

A caveat: this runs after the real decoders, not before. gzip and LZ4 payloads aren't printable either, and you don't want to flag as encrypted something you can actually decompress. Hex strings and hashes need excluding too — they're printable, but they decode to nothing meaningful.

The shape that shows up most often in the encrypted bucket looks like this:

[0x01][key id][ciphertext…]

A version byte, a key identifier, then bytes. Once you've seen it a few times you recognise it instantly, which is itself useful — it means stop, this is not a puzzle.

Giving up is a feature

The UI paints those fields red, with a tooltip that says "likely encrypted / opaque binary". It decodes nothing. It's one of the more useful things in the tool.

Not because it tells you what the data is. Because it tells you to stop looking. Knowing a field is permanently unreadable turns a twenty-minute rabbit hole into a two-second decision to go check something else.

Three rules, then, for this kind of code:

Consume everything or fail. Don't hide a limitation that has a workaround the user could act on. And when you can't read something, say so, instead of producing a structure that looks like an answer.

I build this into Wirepeek, a free DevTools panel for WebSocket traffic — disclosure, it's mine. But the rules are general, and I arrived at all three by getting them wrong first.

The WebSocket frames your DevTools won't decode (and how I caught them)

Iulian Stoian — Mon, 06 Jul 2026 04:59:39 +0000

If you've ever debugged a real-time app — chat, trading, live sports, a collaborative editor — you've opened Chrome DevTools → Network → clicked the WebSocket connection → and stared at this:

↑ 42["chat message","hello"]
↓ 42["chat message","hi there"]
↓ 2
↑ 40

DevTools shows you the frames. What it doesn't do is tell you what they mean.
That 2? It's an engine.io ping. That 40? socket.io CONNECT. The 42[...]?
A socket.io EVENT whose name is the first array element. You learn to read it by eye — 4 = engine.io message, 2 = socket.io EVENT — which is a silly thing to have memorized in 2026.

So I built Wirepeek, a DevTools panel that decodes this stuff live. Here's the interesting part — the bit that turned out harder than I expected.

The naive interceptor is wrong

The obvious way to capture WebSocket traffic from an extension is to replace the constructor:

const Native = window.WebSocket
window.WebSocket = function (...args) {
  const ws = new Native(...args)
  ws.addEventListener('message', spy)
  return ws
}

This misses socket.io traffic, and the reason is subtle. engine.io (socket.io's transport layer) grabs its own reference to the WebSocket implementation early, and depending on bundling and world isolation, your replaced window.WebSocket isn't the one it uses. Replace the constructor and you'll capture some sockets and silently miss others.

The fix is to stop fighting over the constructor and patch the prototype instead — the methods every socket shares, no matter who holds the reference:

const send = WebSocket.prototype.send
WebSocket.prototype.send = function (data) {
  emit('out', this.url, data)        // observe, never mutate
  return send.call(this, data)
}

Outgoing is easy. Incoming is the trap: a page can read messages via addEventListener('message', …) or the onmessage setter, and you have to cover both — while attaching exactly one sniffer per socket so you don't double-count:

const add = WebSocket.prototype.addEventListener
const sniffed = new WeakSet()
function ensureSniffer(ws) {
  if (sniffed.has(ws)) return
  sniffed.add(ws)
  add.call(ws, 'message', (e) => emit('in', ws.url, e.data))
}
WebSocket.prototype.addEventListener = function (type, listener, opts) {
  if (type === 'message') ensureSniffer(this)
  return add.call(this, type, listener, opts)
}
// + override the onmessage accessor on the prototype to call ensureSniffer too

One more thing: this has to run in the page's MAIN world at document_start, before any page script constructs a socket. In an MV3 extension that's a content script with "world": "MAIN". Your chrome.* calls live in a separate ISOLATED-world content script; the two talk over window.postMessage with a namespaced envelope you validate on the other end.

What real sites actually send

I tested on whatever real apps I could find, and the variety was the fun part:

A socket.io chat — clean 42["event",…] frames. Wirepeek shows socket.io · event "chat message" + pretty-printed payload. The thing it was built for.
A sportsbook (SignalR) — {"type":1,"target":"NewLiveOverviewDiffs", "arguments":["…base64…"]}. That's a SignalR hub invocation, and the argument is a base64-encoded, gzip-compressed diff of live odds (so they ship a few hundred bytes per tick instead of full JSON). base64 → gunzip → and out falls readable sportsbook JSON.
Facebook Messenger — this one is a Russian doll: MQTT-over-WebSocket (magic MQIsdp CONNECT), a JSON payload embedded in a binary frame, a base64 field inside that, and inside it a Thrift-Compact sync cursor (last_applied_cursor, the Iris/Delta sync marker). Four layers, no schema.
TradingView — a custom ~m~<len>~m~<payload> framing, and individual frames over 300 KB. Great stress test.

The lesson: the interesting bytes are almost never sitting there as plain JSON — they're base64'd, gzip'd, MessagePack'd, or wrapped in some binary framing. So "pretty-print the JSON, fall back to raw" wasn't enough. It became a decode pipeline you toggle with one switch: deep-nested JSON, JWT, base64, gzip/deflate/LZ4, MessagePack, CBOR, Thrift, Discord's shared zlib-stream — each a small, pure, tested decoder that only fires when it fully consumes the bytes (so no false positives), still falling back to raw for the rest. And when a blob is genuinely encrypted (high non-printable ratio), Wirepeek flags it red instead of pretending — so you stop trying to decode the undecodable. Never throw, never hide a frame.

The bug that taught me MV3

Early on, server-initiated frames at page load (the engine.io open/connect burst) sometimes didn't show up. The capture was fine — the frames were lost downstream. In MV3, the service worker routes frames to the DevTools panel, but if a frame arrives before the panel's port has connected, there's nowhere to send it, and there's no replay. Outgoing frames "worked" only because I emit them synchronously on a user click, by which point the panel was connected.

Fix: buffer per-tab in the service worker and flush the history when a panel connects — capped by both frame count and total bytes, because of those 300 KB TradingView frames. Obvious in hindsight; invisible until you test on a site whose interesting traffic all happens in the first 500 ms.

What it is (and isn't)

Wirepeek is a free, MV3 Chrome extension. Capture is read-only and 100% local — no account, no servers, no telemetry, nothing leaves your machine. It does: live capture (↑/↓, timestamp, size, surviving reloads, iframes too), the decode pipeline above, a collapsible/colored JSON tree with inline image preview, filter/search inside decoded content (substring or regex), timeline markers for navigation and capture start/stop, and copy/export (whole session or a selected range). It speaks 7 languages.

It does not do replay or mocking yet — you inspect your own live traffic, you don't rewrite it. If people actually use this, that's where it goes next.

If you build real-time apps, I'd genuinely like to know which protocol you stare at raw the most. That's the next decoder.

Wirepeek: Chrome Web Store · wirepeek.com