DEV Community

Lakshmi Sravya Vedantham
Lakshmi Sravya Vedantham

Posted on

I Piped My Server Logs Into a Video and Found the Bug in 10 Seconds

#rust #visualization #opensource #cli

What if you could see your data instead of reading it?

I was tailing an access log at 2 AM, trying to find a latency spike buried in 50,000 lines of text. Scrolling. Grep. More scrolling. Then I piped the same log into a tool I'd been building, and the spike jumped out of a heatmap video in ten seconds flat.

That tool is vizpipe.

The Concept

Unix pipes are the most powerful abstraction in computing. You can chain grep, awk, sort, uniq — transforming data through a pipeline that reads left to right like a sentence.

But pipes always end in text. What if they ended in video?

stdin | vizpipe → MP4
Enter fullscreen mode Exit fullscreen mode

That's the entire idea. vizpipe reads numbers from stdin and renders them as an animated video. No browser. No JavaScript. No dashboard. No accounts. Just a pipe and an MP4 file.

5 Commands That Changed How I Look at Data

1. Count to 50, watch it grow 

seq 1 50 | vizpipe --mode bars -o bars.mp4
Enter fullscreen mode Exit fullscreen mode

Each number becomes a bar. The video builds up frame by frame. You see the data accumulate.

2. CSV trend line 

cat sales.csv | vizpipe --mode line --column 2 -o trend.mp4
Enter fullscreen mode Exit fullscreen mode

Extract column 2 from a CSV and draw a self-animating line chart. The line draws itself as data flows in.

3. Log latency heatmap 

cat access.log | vizpipe --mode heatmap --pattern '\\d+ms' -o latency.mp4
Enter fullscreen mode Exit fullscreen mode

Regex extracts millisecond values from log lines. The heatmap lights up — and you see where the spikes cluster.

4. Real-time ping monitoring 

ping -c 20 google.com | vizpipe --mode gauge --pattern 'time=(\\d+\\.?\\d*)' -o ping.mp4
Enter fullscreen mode Exit fullscreen mode

A circular gauge that responds to each ping result. Satisfying and immediately useful.

5. Random scatter 

shuf -i 1-100 -n 200 | vizpipe --mode scatter -o random.mp4
Enter fullscreen mode Exit fullscreen mode

200 random points appearing one by one on a 2D plane. Simple but hypnotic.

How It Works 

stdin --> Parser --> Visualizer --> Renderer --> ffmpeg --> .mp4
           |            |              |
           |            |              +-- PNG frames in temp dir
           |            +-- Trait: bars/line/scatter/gauge/heatmap
           +-- Regex, CSV, JSON, key=value, raw numbers
Enter fullscreen mode Exit fullscreen mode

Three stages:

  1. Parser reads stdin line by line and extracts numeric values. It auto-detects format — raw numbers, CSV, JSON, key=value pairs — or you can specify a regex with --pattern.

  2. Visualizer maintains state and renders each frame showing all data seen so far. This creates the "buildup" animation effect.

  3. Renderer generates PNG frames in a temp directory, then shells out to ffmpeg to encode them as MP4.

No external font dependencies. Text is rendered using a built-in 5x7 bitmap font drawn as rectangles. vizpipe works on any system without font configuration.

5 Visualization Modes

Mode What it does Best for
bars Animated bar chart, color-coded by magnitude Comparing values, rankings
line Self-drawing line graph with fill Time series, trends
scatter Points appear one by one on a 2D plane Distributions, correlations
gauge Circular speedometer Single-metric monitoring
heatmap Color-coded 2D grid Frequency analysis, density

5 Color Schemes

Scheme Vibe
default Blue to red gradient
neon Cyan/magenta/yellow on dark
ocean Cool blues and teals
sunset Warm oranges and purples
mono White on black, clean 
seq 1 100 | vizpipe --mode bars --color-scheme neon --bg-color "#0a0a0a" -o neon.mp4
Enter fullscreen mode Exit fullscreen mode

Data Format Auto-Detection

vizpipe doesn't care what your data looks like. It figures it out.

Format Example Detection
Raw numbers 42, 3.14 Automatic
Space/comma separated 10 20 30 Automatic
CSV name,42,high Use --column N
JSON {"latency": 42} Automatic (top-level numerics)
Key=value cpu=85% mem=72% Automatic
Regex time=42.3ms Use --pattern

Why This Matters

Data visualization shouldn't require:

  • Opening a browser
  • Writing JavaScript
  • Setting up Grafana
  • Creating an account on some SaaS
  • Uploading data to a third party

Sometimes you just want to look at your data. Pipe in, get video out. That's it.

The Unix philosophy says: do one thing well, accept stdin, produce output. vizpipe follows that philosophy — except the output is an MP4 instead of text.

The Stack

Written in Rust using the image and imageproc crates for frame rendering. Video encoding via ffmpeg. 68 tests. Zero runtime dependencies beyond ffmpeg.

Why Rust? Because when you're generating hundreds of PNG frames and encoding video, speed matters. vizpipe processes data and renders frames fast enough that the bottleneck is ffmpeg, not the renderer.

Try It 

# Install from source
git clone https://github.com/LakshmiSravyaVedantham/vizpipe.git
cd vizpipe
cargo install --path .

# Requires: Rust 1.70+, ffmpeg

# Your first visualization
seq 1 50 | vizpipe --mode bars -o first.mp4
Enter fullscreen mode Exit fullscreen mode

Star it if this is useful: github.com/LakshmiSravyaVedantham/vizpipe

Every pipe deserves a picture.

Top comments (0)