I keep quoting one number for my compact feed format: "62% smaller than JSON Feed." It is a convenient midpoint, and honestly a bit of a cop-out, because a single headline figure hides where a compact format actually earns its keep.
Here is the real spread. Same serializer, single-source feed, measured at three sizes:
| Items | JSON Feed | NWF | Reduction |
|---|---|---|---|
| 10 | 5.6 KB | 1.8 KB | 67% |
| 100 | 53.7 KB | 16.8 KB | 69% |
| 1000 | 538.8 KB | 170.6 KB | 68% |
Notice the ratio barely moves as the feed grows. That is the good news.
The catch: those are single-source feeds, and they hit 67 to 69%. Mixed, multi-source feeds only save about 50 to 54%. The "62%" I quote everywhere is just the midpoint between those two cases.
Why the gap? Interning. Authors, tags, and sources are stored once in a table and referenced by index (a:0 means author index 0). If one author wrote forty items, their name appears once, not forty times. A single-source feed repeats itself constantly, so the interning table pays off hugely. A multi-source merge has far less repetition to fold away, so it saves less. The compression is honest about what it is: redundancy removal, not magic.
And here is the part most "I designed a format" posts skip, the part about when not to use it.
If you publish to clients you do not control, keep emitting JSON Feed 1.1. NWF is for machine-to-machine work: syncing feeds between servers, caching large merged outputs, storing a mesh that pulled hundreds of items together. Because the canonical model is the contract, the same feed serializes to both formats, so reaching for the compact one is never a lock-in.
The full design write-up covers the line grammar (one letter per record, TAB-separated cells, LF-separated lines), the byte-for-byte round-trip guarantee, and the complete benchmark method: https://mederic.me/blog/nwf-format-design
If you have ever shipped a compact or binary format: did you publish the honest range, or just the best-case number?
Top comments (0)