DEV Community

RobustTrueTry
RobustTrueTry

Posted on

Your Browser History Search Is Broken—Here's a Local Fix

#docker #search #privacy #productivity

Browser history search has been broken for years. Chrome's omnibox only matches URLs. Firefox's library view chokes on more than a few thousand entries. Cloud-based tools like Memex or Raindrop send every visit to their servers. If you want full-text search over everything you've read—plus your local PDFs and code—without leaving your machine, you need a local indexer.

This article shows how to run Hister, a self-hosted search engine built by the creator of SearXNG. You'll learn the Docker setup, the configuration knobs that matter, and the failure modes that appear once the index grows past a few gigabytes.

The Problem: Fragmented, Limited, or Leaky

Most developers cobble together three half-solutions:

  • Browser history: limited to 90 days in Chrome, no full-text, no file content.
  • Spotlight / Windows Search / locate: indexes files but not browser visits, and misses content inside PDFs or code repositories.
  • Cloud notebooks: Notion, Obsidian Sync, Readwise—convenient but they own your data.

Hister merges browser history and filesystem content into one searchable index. It runs locally, exposes a REST API, and stays off the network unless you proxy it.

The Approach: Index What You Already Have

Hister watches two sources:

  1. Browser databases — SQLite files from Chrome, Firefox, Brave, Edge. It reads History, Visited Links, and Top Sites tables.
  2. Filesystem paths — any directory you point it at. It extracts text from PDFs, Office docs, source code, markdown, and plain text using Apache Tika.

The index lives in a local Meilisearch instance. Queries hit Meilisearch directly; Hister only handles ingestion and deduplication.

Implementation: Docker Compose Stack

Create a docker-compose.yml that mounts your browser profiles and target directories read-only. This example assumes Linux paths; adjust for macOS (~/Library/Application Support/...) or Windows (%LOCALAPPDATA%/...).

version: "3.9"
services:
  meilisearch:
    image: getmeili/meilisearch:v1.11
    volumes:
      - meili_data:/meili_data
    environment:
      MEILI_NO_ANALYTICS: "true"
      MEILI_ENV: "production"
    restart: unless-stopped

  hister:
    image: asciimoo/hister:latest
    depends_on:
      - meilisearch
    volumes:
      - ./config.yml:/config.yml:ro
      - /home/user/.config/google-chrome/Default:/chrome:ro
      - /home/user/.mozilla/firefox/xxxx.default-release:/firefox:ro
      - /home/user/Documents:/docs:ro
      - /home/user/Code:/code:ro
    environment:
      HISTER_CONFIG: /config.yml
    restart: unless-stopped

volumes:
  meili_data:
Enter fullscreen mode Exit fullscreen mode

The config file tells Hister which sources to index and how often:

meilisearch:
  url: http://meilisearch:7700
  index: hister

sources:
  - name: chrome-history
    type: chrome
    path: /chrome/History
    interval: 300
  - name: firefox-history
    type: firefox
    path: /firefox/places.sqlite
    interval: 300
  - name: documents
    type: filesystem
    path: /docs
    interval: 3600
    extensions: [pdf, docx, txt, md, py, js, ts, rs, go]
  - name: code
    type: filesystem
    path: /code
    interval: 3600
    extensions: [py, js, ts, rs, go, java, cpp, h, rs]

## Optional: drop visits older than 2 years to bound index size

retention:
  max_age_days: 730
Enter fullscreen mode Exit fullscreen mode

Run docker compose up -d. The first index run takes 10–30 minutes depending on history size. After that, Hister polls each source at its interval (seconds) and increments the Meilisearch index.

What Breaks at Scale

Once the index grows past a few gigabytes, three practical problems appear. Each has a straightforward fix.

Meilisearch memory grows with unique terms

Meilisearch keeps the full inverted index in RAM. A 5 GB history + docs corpus can push the container past 4 GB RSS. Set a hard limit:

    deploy:
      resources:
        limits:
          memory: 6G
Enter fullscreen mode Exit fullscreen mode

If you hit OOM kills, reduce indexed fields. Edit the Meilisearch index settings via API:

curl -X PATCH 'http://localhost:7700/indexes/hister/settings' \
  -H 'Content-Type: application/json' \
  -d '{"searchableAttributes": ["title", "url", "content"]}'
Enter fullscreen mode Exit fullscreen mode

Dropping content from searchable attributes saves ~40% RAM but loses full-text search inside documents.

Browser databases lock during writes

Chrome holds a lock on History while running. Hister opens it read-only, but long transactions can still block the browser. The workaround: copy the file before reading.

Add a small wrapper script to your compose file:

  hister:
    # ...
    command: ["/bin/sh", "-c", "cp /chrome/History /tmp/History && exec hister"]
    volumes:
      - /home/user/.config/google-chrome/Default:/chrome:ro
      - /tmp:/tmp
Enter fullscreen mode Exit fullscreen mode

Update the config path to /tmp/History. This adds ~2 seconds per poll cycle and eliminates lock contention.

Filesystem polling misses rapid changes

The interval setting is a floor, not a ceiling. If you save a file, it won't appear in search until the next poll. For code directories, consider a sidecar watcher:


## Run on host, not in container

while inotifywait -r -e close_write /home/user/Code; do
  curl -X POST http://localhost:7700/indexes/hister/documents \
    -H 'Content-Type: application/json' \
    -d @<(hister-extract /home/user/Code/changed_file.py)
done
Enter fullscreen mode Exit fullscreen mode

hister-extract is a tiny CLI (included in the image) that outputs a single document JSON. This keeps the index near-real-time for active projects without polling the whole tree every minute.

Duplicate URLs across browsers

If you use Chrome and Firefox, the same page appears twice. Hister deduplicates by URL within a source, not across sources. Add a post-ingestion dedupe job:


## dedupe.py — run daily via cron

import requests
import json

resp = requests.get('http://localhost:7700/indexes/hister/documents',
                    params={'limit': 10000})
docs = resp.json()['results']
seen = {}
for d in docs:
    key = d['url']
    if key in seen:
        requests.delete(f'http://localhost:7700/indexes/hister/documents/{d["id"]}')
    else:
        seen[key] = d['id']
Enter fullscreen mode Exit fullscreen mode

Run it after the nightly index window. It keeps the newest document ID per URL.

Tradeoffs at a Glance

Approach Privacy Full-Text Setup Effort Maintenance
Chrome/Firefox built-in Local only No Zero None
Spotlight / Windows Search Local only Files only Zero OS updates
Cloud (Readwise, Memex) Vendor sees all Yes Low Subscription
Hister + Meilisearch Fully local History + files Medium Index pruning, RAM

Choose Hister when you need unified search across browser visits and local files, and you're willing to operate a small stack. Skip it if you only need file search—ripgrep + fzf is faster and lighter.

Key Takeaways

  • Run Hister behind a reverse proxy (Caddy, Traefik) with basic auth if you expose it beyond localhost.
  • Monitor Meilisearch RAM; set a container limit and trim searchableAttributes before it OOMs.
  • Copy browser SQLite files to /tmp before indexing to avoid locking the browser.
  • Add a nightly dedupe job if you index multiple browsers.
  • For code directories, pair polling with an inotifywait sidecar for near-real-time updates.

Source

Hister: A private search engine for the pages you visit and the files you keep — this article adds a production Docker Compose stack, Meilisearch memory tuning, browser lock workaround, cross-browser deduplication script, and a tradeoff table the README does not cover.

Support this work

These write-ups are researched and published with no paywall, sponsor, or tracking. If one saved you an afternoon, a small tip keeps them coming.

USDT, USDC or USDD · TRC-20 (Tron)

TFTNsfyomKrnUutRjBTGVULp19ByW29KbY
Enter fullscreen mode Exit fullscreen mode

Top comments (0)