The design of my large-file text viewer UwView traces back to the old UwView I distributed on Vector back in the Windows 95 era — 30 years ago. I rebuilt it from scratch in C#/Avalonia, and released a premium edition, UwView Pro (Windows/macOS/Linux; one-time $129 or $9/month; product page https://uvp.y42u.net/en/pro-en/). Now that I have solid measurements from the core engine, here's a serious head-to-head against klogg, the industry-standard viewer for huge log files.
Cutting straight to the conclusion: when actually measured, klogg and the current (free) UwView performed at roughly the same level. In other words — isn't the "industry standard" older than you'd expect? This article is about that, and about what happened when I redesigned that same architecture from scratch for today's hardware.
Test conditions — a real 48GB file, 892,239,125 lines
All benchmarks were run under identical conditions on the identical file.
- Target file: OSM data for all of Japan,
japan-latest.osm(47.73GB, 892,239,125 lines — the same real-world data used in my earlier 51GB experiment)
Note on file size notation: the "48GB" figure here is 47.73GiB (bytes ÷ 1024³), rounded. macOS's own size display (SI units, ÷1000³) shows 51.25GB. Both refer to the same file (51,254,526,392 bytes).
- Environment: 10-core Mac, 32GB RAM, external USB drive (measured physical throughput 0.41GB/s)
- Comparison target: klogg 24.11.0 (main search always uses the regex engine, Match case ON)
- Search hit counts were confirmed to match exactly across all patterns among klogg, UwView, and UwView Pro (e.g., literal "Tokyo" = 10,967 hits)
Round 1: klogg vs. current UwView — nearly identical
| Item | klogg 24.11 | UwView (public release) |
|---|---|---|
| First open (index build) | About 110 seconds | 128–186 seconds |
| Every subsequent open | Every time, about 110 seconds (re-indexes) | Every time, 128–186 seconds (rebuilds) |
| Search literal "Tokyo" | 120–135 seconds | 156.2 seconds |
There's some variance in the numbers, but it's clear that both tools are built on the same principle: "read the entire 48GB raw file, every single time." Both hover around 0.3GB/s throughput — meaning both are pinned to the physical bandwidth ceiling of the disk. That's not a matter of which tool is better; it's a matter of which design generation each belongs to.
To be clear, I did not look at klogg's source code at all. This is an inference based purely on external measurements hitting the same physical ceiling, but the underlying structure — rebuilding the index every time you open the file, and scanning the raw file every time you search — appears to belong to the same design generation.
And current UwView's design originates from the old UwView of 30 years ago. The first discovery from this comparison: the industry-standard klogg performs on par with a 30-year-old design.
One point where it isn't even "on par"
One feature the old UwView has had consistently since 30 years ago is that "the entire file is visible the moment you open it" (progressive open). Current UwView also lets you jump freely to the middle or end of the file even while the index is still being built.
What about klogg? During my measurements, I confirmed that only the beginning of the file is visible until index building finishes (you cannot scroll to the middle or end). For a 48GB file, that's roughly 110 seconds during which the end of the file is simply off-limits. On this one point, klogg doesn't even match the 30-year-old design — it falls short of it.
Round 2: Enter UwView Pro — a design rebuilt for "now"
What's changed between 30 years ago and today? RAM in the tens of gigabytes is now standard, CPUs commonly have 10 cores, and compression like zstd — "compression faster than the disk itself" — is now available. UwView Pro's engine was redesigned from the ground up around these assumptions.
At the core is a sidecar approach. Three key points:
- On the first open, a zstd-compressed cache is generated at the same time the index is built (compression happens on otherwise-idle CPU cycles between disk reads, so it adds essentially zero extra wait time)
- From the second open onward, only the saved index needs to be read (milliseconds). Searches read the 5.3GB compressed cache instead of the 48GB raw file (a 1/9 reduction in bytes read)
- The compressed cache preserves every byte of the original file losslessly, with block-level checksums. Even if you delete the original file and keep only the 1/9-sized cache, you can still open and search it directly
Overall comparison table — measured results
| Item | klogg 24.11 | UwView (public release) | UwView Pro (on sale) | Pro ÷ klogg |
|---|---|---|---|---|
| First open | ~110 s (only the beginning visible until complete) | 128–186 s (entire file visible immediately) | 140 s (includes index + compressed cache; entire file visible immediately) | About the same |
| Every subsequent open | ~110 s every time | 128–186 s every time | 0.02–0.07 s | 1,500x or more |
| Search literal "Tokyo" | 120–135 s | 156.2 s | 14.3 s | About 9x |
| Search case-insensitive "Tokyo" | ~120 s | (not measured) | 14.0 s | About 8.6x |
| Search regex "Tok[yi]o" | ~130 s | (not measured) | 29.8 s | About 4.4x |
| Search regex "name:en.*Tokyo" | ~130 s | (not measured) | 29.2 s | About 4.4x |
| Disk footprint (archival use) | 48GB (original required) | 48GB (original required) | 5.3GB (original deletable; checksum-protected) | 1/9 |
An honest note
- Instant to view from the very first open. UwView lets you read, scroll and search the entire text the moment you open it (klogg shows only the head until indexing finishes). The index and compressed cache are built on the first open, in the background (about 140 s for 48GB — bound by reading the raw file once at 0.41GB/s; you can browse while it runs). Because indexing happens on the first open, from the second open on it's instant, with line numbers (0.02–0.07 s).
- All figures above are measurements from a single real-world environment. RAM capacity, storage speed, and file contents will all affect the numbers.
Current status
The engine (parallel index building, parallel search, and the sidecar mechanism) is fully implemented, and every figure above comes from measurements of working code. UwView Pro is on sale now (Windows/macOS/Linux; one-time $129 or $9/month). → https://uvp.y42u.net/en/pro-en/
Current UwView (free) remains available on GitHub. Give the "entire file visible the moment you open it" experience a try — it's been a core part of UwView for 30 years.
Summary
- Measured head-to-head, the industry-standard klogg and current UwView (built on a 30-year-old design) perform at roughly the same level — both belong to the "read the entire raw file every time" generation.
- On "the entire file is visible the moment you open it," klogg doesn't even match the 30-year-old UwView (only the head is visible until indexing completes).
- UwView Pro is a next-generation engine redesigned for today's hardware (abundant RAM, multi-core CPUs, compression faster than disk). Measured on 48GB / 892M lines: 1,500x+ for subsequent opens, ~9x for search, 1/9 the storage.
- UwView Pro is on sale now (Windows/macOS/Linux; one-time $129 or $9/month). → https://uvp.y42u.net/en/pro-en/
Related articles
- UwView Released on GitHub — Renewing a Windows 95-Era Large-File Text Viewer
- I Opened a Real 51GB, 892,239,125-Line Dataset (OSM Japan) in UwView
- How Many Lines Can UwView Theoretically Open?
Sources
- amru195704/UwView (GitHub) https://github.com/amru195704/UwView
- klogg (GitHub) https://github.com/variar/klogg
- OpenStreetMap data (Geofabrik: japan-latest.osm) https://download.geofabrik.de/asia/japan.html
- Zstandard (zstd) https://github.com/facebook/zstd
From the developer: a list of my apps, Kindle books and open-source projects is on GitHub: amru195704.
Top comments (0)