DEV Community

y4u
y4u

Posted on Originally published at uvp.y42u.net

From 100,492 Hits to 3 in Two Clicks — Drill-down Search for Huge Local Files

UwView Pro just gained Drill-down Search: refine your search results with another term, then another — up to 8 stages, each in its own tab.

Drill-down search: New York 100,492 → Central Park 226 → coffee_shop 3

It's the grep pipeline you already type every day

When you dig through a big log on the command line, you write something like:

grep -C 5 "ERROR" huge.log | grep "timeout" | grep "db-server3"
Enter fullscreen mode Exit fullscreen mode

"Find ERROR, keep 5 lines of context, then keep only the chunks that also mention timeout, then db-server3." Drill-down Search is exactly that pipeline — as GUI tabs.

Run a normal search and the results pop up as usual (same as the free edition). In Pro, the popup has a refine bar: type the next term, hit Refine, and a new tab appears containing only the matches found within the ±N context of the previous stage. Each tab shows term (count), so you can watch the numbers collapse.

Real example: finding one coffee shop in 258 GB / 4.5 billion lines

The cover image is the entire US OpenStreetMap extract — one XML file of 258.68 GB and 4,509,830,821 lines, opened directly from its compressed .uwvz archive without unpacking:

New York (100,492) → Central Park (226) → coffee_shop (3)

"New York" alone: 100,492 hits in 30.28 s. Refine with "Central Park": 226. Refine with "coffee_shop": 3. Two refinements took 100k hits down to three — and hit #1 lands on a Starbucks at Central Park South (59th & Broadway branch), with its address, opening hours and latitude/longitude sitting right there in the raw tags. One coffee shop, surfaced out of 4.5 billion lines.

Three things worth noting:

  • Stages after the first are instant. They only scan the previous stage's context blocks, so the only real wait is the initial full-text search.
  • ±N context is set per stage. Keep intermediate stages tight, then open the final stage up (the screenshot uses ±12) to actually read the story.
  • Every tab keeps the usual tools: jump to the source line, save results to a file — and a right-click History (drill-down path) menu shows exactly which hits led you here, line numbers included.

How this differs from klogg's Boolean search

klogg's and/or Boolean search works within a single line. Drill-down finds co-occurrence across nearby lines — and in real logs, the clues are almost never on one line.

The same tabbed UI is also gaining Sequence Search — detecting "w1 → w2 → w3 in that order," the desktop equivalent of Elastic EQL's sequence or Splunk's transaction. Almost nothing offers that against a multi-gigabyte local file. More on that soon.

Get it

Drill-down Search ships in UwView Pro V1.3 — released and available now. The free edition keeps the classic single-stage search with ±1 context and result export.

UwView Pro V1.3 is available on Polar. The monthly subscription comes with the first month free (the one-time purchase has no free trial).

Related

Top comments (0)