y4u

Posted on Sep 16 Originally published at uvp.y42u.net

Beating ripgrep Even at 3 GB — Concede the Single grep, Win the Rest of the Investigation

#ripgrep #cli #productivity #showdev

A grep on 3 GB belongs to ripgrep.

In the previous article I wrote that 3 GB is ripgrep's ground and I concede it. When the file fits in memory, an index has little to offer. So is 3 GB a loss no matter what? I went back through the same measurements and read them from the other side — not the rows we lost, but the rows we won.

Up front

I compared ripgrep / sed / gzip against uvp on a 3 GB file (3,032,812,644 bytes cut from OpenStreetMap Japan XML) under the same rules (Apple M4 / 32 GB / external SSD, sudo purge before every run, /usr/bin/time -p real, uvp on its second run = index present).

Pure search: ripgrep wins every row (0.33 s vs 0.75 s = 1/2.3). The reason is structural — uvp's fixed cost of 0.6–0.7 s for startup plus index load is larger than rg's whole scan at 0.33 s. The absolute gap is 0.4 s. Those are warm-cache figures, though: right after sudo purge it flips to rg 3.23 s vs uvp 1.01 s (1/12 the bytes read)
Even so, there are four situations where you win at 3 GB: ① fixing and saving (replace 3.5× and extract 12.6× over sed), ② operations rg does not have (sequence detection, ±N-line drill-down, click-to-drill), ③ logs kept as gz/zip (expand + rg 6.47 s vs .uwvz direct 0.68 s = 9.5×), and ④ not yet knowing what you are looking for (narrow in the CLI, look in the GUI, narrow again — 0.24 s per stage)
In short: on a day that ends with grep, use rg. The moment the investigation involves reading, narrowing, counting, fixing or saving, 3 GB becomes uvp's ground too

First, lock in the loss

Pure search at 3 GB, second run (index present). ripgrep is faster on every row.

Test	`rg`	`uvp`	rg ÷ uvp
Search (fixed string)	0.33 s	0.75 s	1/2.3
Case-insensitive	0.55 s	0.72 s	1/1.3
Regex	0.54 s	0.80 s	1/1.5
Two-term narrowing	0.50 s	0.80 s	1/1.6
Tally (top 20)	0.45 s	0.83 s	1/1.8
First 10 hits	0.07 s	0.25 s	1/3.6
Write to file	0.34 s	0.86 s	1/2.5

The reason is clear. 3 GB fits in 32 GB of RAM, so after one read the whole file sits in the OS cache. rg running straight over that cache takes 0.33 s. uvp launches its engine and then loads the index, so its 0.6–0.7 s fixed cost comes first, and that is where it loses. The smaller the data, the larger the share of that fixed cost. It is the nature of an index-based approach and there is no hiding it.

I concede this. The gap is 0.4 s, and conceding it costs nothing in practice.

(There was exactly one exception: for a regex with no extractable required literal, such as [0-9]{3}-…, rg drops to 2.59 s and uvp's 1.08 s wins. Half of that is a weak spot on rg's side, so I do not count it as a winning pattern.)

But that loss only holds while the cache is warm

The table above is "second run, warm cache". I re-measured the same search starting right after dropping the page cache with sudo purge (3 runs each, median).

	Bytes read	Cold (right after purge)	Warm (again immediately)
`rg -c 東京` (3 GB)	3,032,812,644	3.23 s	0.25 s
`uvp` (`.uwvz`)	251,829,224	1.01 s	0.71 s
rg ÷ uvp		3.20× (uvp)	1/2.84 (rg)

Cold, uvp is 3.2× faster even at 3 GB. rg reads 3 GB from disk every time (3.23 s); uvp reads 0.25 GB. Once warm, rg drops to 0.25 s while uvp keeps its startup cost, and the order flips — the table of losses is a snapshot of that "after it warmed up" state only.

The same measurement at 10 GB shows one more thing.

10 GB	Cold	Warm
`rg -c 東京`	10.91 s	10.91 s
`uvp` (`.uwvz` 1.16 GB)	1.91 s	1.45 s

At 10 GB there is no such thing as a warm rg. The file does not fit in 32 GB of RAM, so the second run is the same 10.91 s. "At 3 GB, use rg" holds only when the file fits in memory and someone read it a moment ago. Right after a reboot, on a log you last opened days ago, on a USB HDD or a network share, the difference in bytes read shows up even at 3 GB.

The problem is that an investigation almost never ends after one grep.

① A day you fix and save — when the job writes output, bytes read matter directly

Search is "read and count". Extract and replace are "read and write out". That changes things.

Test	CLI	`uvp`	Ratio
Extract hit lines (`sed -n '/term/p'`)	9.96 s	0.79 s	12.6×
Replace all (`sed 's/A/B/g'`)	15.04 s	4.28 s	3.5×
Same (`rg --passthru -r`)	—	4.28 s	1.7×

sed processes one line at a time, so even 3 GB takes 10–15 s. uvp pulls only the lines it needs out of a 0.25 GB index and writes them, so it wins even at 3 GB. The replace output is byte-identical to sed and to rg --passthru -r in every combination tested.

Add up a day's flow — "investigate, fix, save" — and it looks like this.

Open → 2 queries → replace and save (3 GB)	Total
rg + sed	17.7 s
uvp (including index build)	7.6 s (2.3×)

uvp's 7.6 s includes 1.2 s to build the index on the first run. It still wins. One replace turns sed's 15 s into uvp's 4.3 s, and next to that 10-second gap the 1.2 s for the index is noise.

② A day you use something rg does not have

Some rows could only go into the comparison as "reference", because rg has no equivalent command.

Operation	`uvp` (3 GB)	In rg you would…
Sequence detection `-seq login,timeout,restart` (only flows in this order)	0.99 s	write a state machine in awk
Second term within ±N lines `-C 3 <term2>` (co-occurrence nearby)	1.23 s	`rg -C 3 \
Click a tally value to drill into it	instant in the GUI	read {% raw %}`rg -o \
Jump from a hit to the original text with line numbers	instant in the GUI	note the line number from {% raw %}`rg -n`, then `sed -n 'Np'`

Before "faster or slower", these are rows where the comparison itself does not exist. "Only the flows where a timeout follows a failed login within three lines", "only the hits with another term within five lines" — use any one of these in an investigation and whether the file is 3 GB stops mattering.

③ A day the logs are kept as gz / zip — stop expanding them every time

Logs are normally kept gzipped. To search them you expand and then rg — and zgrep does not save you the expansion itself.

I compressed the same 3 GB with gzip -6, zip -6 and .uwvz, and compared expand-then-search with search directly.

3 GB	Time to compress	Size	Search (expand + rg / direct)
gzip -6	16.8 s	0.30 GB (1/10.1)	6.47 s (expand 3.49 + rg 2.98)
zip -6	19.4 s	0.30 GB (1/10.1)	8.71 s (expand 8.08 + rg 0.63)
`.uwvz`	1.2 s	0.25 GB (1/12.0)	0.68 s (direct search)

Same size as gzip, 14× faster to create, 9.5× faster to search. gzip expands everything every time you search; .uwvz carries an index and is searched without expanding. The second and third questions stay at 0.68 s. You could keep the expanded file around and get rg's 0.33 s from the second question on, but then there was no point compressing.

And .uwvz restores the original byte-for-byte with -extract (free). Change the storage format from gz to .uwvz and the per-search expansion disappears, even at 3 GB. At 50 GB it is 7 s against gzip's 2 minutes — 17×.

④ A day you do not yet know what you are looking for — this is the real point

In the table of losses, 東京 at 3 GB returns 11,274 hits. rg's 0.33 s does not include the time for a person to read those 11,274 lines.

There are two kinds of investigation. Checking whether a string is there — and the kind where you do not yet know what to search for. The second is trial and error itself: scan the 11,274, look around the interesting ones, narrow by another term, count, go back. In a terminal you rebuild the command each time for 0.33 s — except that what you actually pay is the time to rebuild it and the time to follow output scrolling past.

Put -open at the end of a uvp command and the CLI result is handed straight to the GUI.

uvp japan-dv-ai.uwvz '東京' -uniq 'k="([^"]+)"' -sort count -head 20 -open

The CLI finishes the search and the tally, and hands the result to the GUI through a temporary file (the GUI does not search again; v1.6.2)
The GUI reads the same .uwvz, so reopening does not read 3 GB (reopening a 47.73 GB file takes 0.02–0.07 s from the second time on; 3 GB is less)
Click a row in the list to jump to it in the text. ±N lines of context. Click a value in the tally to drill into it as the next stage
Narrow again with another term — 0.24 s per stage (measured at 50 GB; 3 GB is less)
Once you know what you are after, drop back to the command line and process in bulk. The GUI and the CLI use the same .uwvz, so moving between them costs no waiting

So the thing to compare is not "one grep, 0.33 s vs 0.75 s" but "total time and number of actions to reach the answer". rg alone means rebuilding the command, reading the output, rebuilding again for every re-narrowing. GUI alone means a slow open. Put an index between them so the CLI and the GUI use the same thing, and narrow broadly with a command, look closely on screen, drop back to a command connects with no waiting.

The CLI and the GUI call the same functions (drill-down, tally, sequence), so the count the CLI prints and the count the GUI shows always agree. That was verified byte-for-byte against rg / sed across 81 combinations plus 12 gz/zip inputs plus every 258 GB case, with zero defects on the product side.

Summary — four situations where you win at 3 GB

Situation	Measured at 3 GB
A day that ends with grep	rg wins. 0.33 s vs 0.75 s. I am not going to try to close that gap (though with a cold cache it flips: 3.23 s vs 1.01 s)
① A day you fix and save	Replace 3.5×, extract 12.6× over sed. 2.3× on "open → 2 queries → replace and save"
② A day you need something rg lacks	Sequence detection, nearby co-occurrence, click-to-drill, line-number jump. No comparison exists
③ A day the logs are gz/zip	Expand + rg 6.47 s vs direct 0.68 s (9.5×). Compressing takes 1/14 of gzip's time
④ A day you do not yet know what you are looking for	Narrow in the CLI, look in the GUI, narrow again. 0.24 s per stage. The unit of comparison changes from "one grep" to "time to the answer"

"A single grep on 3 GB" is ripgrep's ground, and it can be conceded. The moment an investigation involves any one of reading, narrowing, counting, fixing or saving, it becomes uvp's ground — and past 10 GB, pure search wins by 5–10× as well.

The tools

uvp ships with UwView Pro v1.6.2 (Windows, macOS and Linux, one licence for all three; one-time or monthly; 14-day free trial, during which uvp and editing both work). Replace (-replace) is part of the Edit Upgrade. Restoring from .uwvz (-extract) is free. The free UwView ships uvf, which does search and -open only and has no index.

All figures are my own measurements on my own machines, not an independent benchmark. ripgrep is among the fastest there is for searching across many files, and this article does not contest that. If you spot a configuration issue, tell me; I will check and correct.

Conditions

Apple M4 / 32 GB / macOS 26.3.1 / external SSD
ripgrep 15.2.0 / BSD sed / gzip and zip at level 6 / uvp 1.6.1–1.6.2 / sudo purge before every run, cooldown between runs
3 GB: 3,032,812,644 bytes cut from OpenStreetMap Japan XML. The .uwvz is 251,829,224 bytes (1/12.0); gzip is 300,890,910 bytes (1/10.1)
Table values are second runs (.uwvz reused). The first run includes 1.2 s of index build, 1.96 s in total (rg 1.66 s; 1/1.2)
Cold/warm measured 2026-09-16 with uvp_cold_test.sh, two sessions of 3 runs each (6 in total), median. The two sessions agree at 3 GB: rg cold 3.23 / 3.24 s, uvp cold 1.01 / 1.05 s. uvp user time 3.6–3.9 s (decompressing across cores)

DEV Community