Someone hands you a log for an incident investigation — and it's 10 GB, sometimes 50 GB. Double-clicking does nothing. Your editor freezes. The investigation stops before it starts.
Why editors give up
Notepad dies at a few hundred MB. VS Code and Notepad++ freeze or drop into a restricted mode at a few GB. The reason is structural: an editor loads the whole file into memory so you can edit it. A 50 GB log will never fit in 32 GB of RAM. This isn't a bug you can configure away — it's what editing requires.
Why "just split it" backfires
The standard advice is split -b 1G. In a real investigation, it makes things worse:
- Line numbers are gone — you can't report "the error is on line N" anymore
- Context is severed — what happened right before the error now lives at the tail of a different file
- 50 GB = 50 files to open one by one. Search becomes manual labor again
What you need is not fragments. It's one log, viewable as one log.
The workflow (free)
UwView is a free viewer for huge text files (Windows / macOS / Linux). It can't edit — in exchange, it starts displaying before the file is fully read.
- Download (Windows: unzip and run, no installer)
- Drag & drop the log
- Scroll immediately. Drag the scrollbar to the bottom — you're at the newest entries
- Search "ERROR" from the toolbar (regex supported)
- Jump to a hit from the results list, read the surrounding lines in place
Indexing runs in the background; line numbers appear when it's done. You don't wait for it.
One measured data point: a 47.73 GB, ~892-million-line file (OSM data, USB SSD, 32 GB-RAM machine) — scrolling to the end and searching worked immediately after opening. The line-count ceiling is ~550 billion lines; in practice your disk runs out first.
Honest scope note
A one-off investigation — open once, find the error, read the context — is fully covered by the free edition, including regex, color rules, and encoding auto-detection (CJK included).
Two waits remain in the free edition: the index is rebuilt on every reopen, and every search reads the file itself. If you keep returning to the same logs, UwView Pro (v1.3.0+) saves the index and a compressed cache: reopen in ~0.02–0.07 s, search ~9× faster, and archive at ~1/9 size that opens without unpacking. For a one-time job, the free edition is enough — that's the honest split.
Full walkthrough: the original article. Source: GitHub.
Figures are measurements in one environment and will vary.
Top comments (0)