DEV Community

the bomber
the bomber

Posted on

Capturing a Screen Region with PipeWire and Extracting Text via Tesseract

#linux #ocr #automation

Hook: Stop Copy‑Pasting Numbers from a Dashboard with No API

You stare at a web‑based dashboard, manually copy‑pasting a handful of numbers every hour. The page has no API, no export button, and the data changes fast enough that a manual workflow becomes a bottleneck. What if you could let your Linux box watch a specific region of the screen, turn the pixels into text, and trigger an alert the moment a value changes?

In this post I walk through the exact setup I use on my home server: a PipeWire screen capture, a tiny grim script to grab a region, Tesseract OCR to read the numbers, and a Bash loop that notifies me via notify-send when the value differs from the previous run.

1. Prerequisites

Tool Why you need it
pipewire (with pw-cli and pw-top) Modern screen‑capture backend on Wayland
grim Simple command‑line screenshot utility that works with PipeWire
tesseract-ocr OCR engine that can read the captured image
imagemagick (optional) Pre‑process the image for better OCR accuracy
jq (optional) Parse JSON if you decide to log results
notify-send (or any other notifier) Show a desktop notification when the value changes

All of these packages are available in the default repositories of most distros:

# Debian/Ubuntu
sudo apt install pipewire grim tesseract-ocr imagemagick jq libnotify-bin

# Fedora
sudo dnf install pipewire grim tesseract imagemagick jq libnotify
Enter fullscreen mode Exit fullscreen mode

2. Capture a Fixed Region with grim

grim can capture the whole screen or a rectangle defined by geometry. I store the geometry in a variable so I can tweak it without editing the script.

# Geometry: X,Y,W,H (pixels)
REGION="1200,300,250,80"   # Adjust to the exact location of the number on your dashboard
IMG_PATH="/tmp/dashboard.png"

capture_region() {
    grim -g "$REGION" "$IMG_PATH"
}
Enter fullscreen mode Exit fullscreen mode

Run grim -g "$REGION" - to preview the region in a viewer like feh if you need to fine‑tune the coordinates.

3. Pre‑process the Image (Optional but Recommended)

Tesseract works best with high‑contrast, binarized images. A quick ImageMagick pipeline removes noise and boosts contrast.

preprocess_image() {
    convert "$IMG_PATH" \
        -colorspace Gray \
        -level 20%,80% \
        -type Bilevel \
        "$IMG_PATH"
}
Enter fullscreen mode Exit fullscreen mode

If the dashboard already uses a clean font on a solid background you can skip this step.

4. Run Tesseract and Extract the Number

We only need the plain text output, so we tell Tesseract to write to stdout (-). The --psm 7 mode treats the image as a single line of text, which speeds up processing.

extract_number() {
    tesseract "$IMG_PATH" stdout --psm 7 2>/dev/null | tr -d '\n' | tr -d ' '
}
Enter fullscreen mode Exit fullscreen mode

The tr commands strip newlines and spaces, leaving a compact string like 42.7.

5. Detect Changes and Notify

The core loop stores the previous reading in a temporary file. When the new reading differs, we fire a notification.

STATE_FILE="/tmp/dashboard_last.txt"

check_change() {
    local current=$(extract_number)
    local previous=""
    [ -f "$STATE_FILE" ] && previous=$(cat "$STATE_FILE")

    if [[ "$current" != "$previous" ]]; then
        notify-send "Dashboard Update" "Value changed: $previous → $current"
        echo "$current" > "$STATE_FILE"
    fi
}
Enter fullscreen mode Exit fullscreen mode

6. Glue It All Together

A simple infinite loop with a sleep interval does the job. I run this as a systemd user service so it survives reboots and logs to journalctl.

#!/usr/bin/env bash

while true; do
    capture_region
    preprocess_image   # comment out if you skipped pre‑processing
    check_change
    sleep 60   # check once per minute; adjust as needed
done
Enter fullscreen mode Exit fullscreen mode

Save the script as ~/bin/dashboard-watcher.sh, make it executable (chmod +x), and test it manually before creating the service.

7. Optional: Systemd User Service

Create ~/.config/systemd/user/dashboard-watcher.service:

[Unit]
Description=Watch dashboard region and alert on change
After=network.target

[Service]
ExecStart=%h/bin/dashboard-watcher.sh
Restart=on-failure

[Install]
WantedBy=default.target
Enter fullscreen mode Exit fullscreen mode

Enable and start it:

systemctl --user daemon-reload
systemctl --user enable --now dashboard-watcher.service
Enter fullscreen mode Exit fullscreen mode

You can view logs with journalctl --user -u dashboard-watcher.service.

8. Tweaking for Accuracy

  • Font size & DPI – Larger fonts and higher DPI screenshots improve OCR confidence.
  • Whitelist characters – If you only expect numbers and a decimal point, add tesseract config tessedit_char_whitelist 0123456789.
  • Language data – Use tesseract-ocr-eng (default) unless the dashboard uses a different script.

9. What This Solves

  • Zero manual copying – The script runs unattended.
  • Immediate alerts – Get a desktop pop‑up (or replace notify-send with an email/SMS hook) the moment a KPI shifts.
  • Low overhead – Each cycle consumes < 0.2 s of CPU on a modest VM.

You can extend the pattern to multiple regions, feed the numbers into a time‑series database, or trigger a Home Assistant automation.

I build sites and automations for small businesses at dreamfuturestech.com.

Top comments (1)

Collapse
 
autenai profile image
Auten •

Nice minimal loop. A few things that bit me with the same setup:

  • grim doesn't go through PipeWire: it talks to the compositor directly via the wlroots screencopy protocol. So it works on Sway/Hyprland, but on GNOME it just fails. The PipeWire route on GNOME/KDE is the xdg-desktop-portal ScreenCast API, which shows a consent dialog the first time, so it's worth saying which compositor you run.
  • As a systemd user service, grim often can't find the display because the unit doesn't inherit WAYLAND_DISPLAY. systemctl --user import-environment WAYLAND_DISPLAY (or setting it in the unit) fixes it.
  • For numbers, -c tessedit_char_whitelist=0123456789.- plus upscaling the crop 2-3x before binarizing removes most misreads (a 5 read as an S, a stray space in the middle).
  • Fixed geometry breaks when the window moves or the browser zoom changes. If the dashboard is a web page, reading the value from the DOM with a headless browser is sturdier than OCR; keep OCR for native apps that expose nothing.

Do you get false "changed" alerts from OCR noise, or is the font clean enough that it's stable?