I Had to Scrape My Own Reading History. So I Built a Self-Hosted Alternative.

#opensource #showdev #python #books

For years, I used an Android app called Dante to track my reading.

It wasn't flashy. It didn't have social features, tracking telemetry, or algorithmic recommendations. It simply let me scan a book, track my page progress, and keep a historical log of what I'd read.

And for a long time, that was enough.

Then one day, it started breaking. The external search API the app relied on went silent. ISBN lookups stopped returning data. Searching by title threw errors. Features that depended on external services slowly became completely unusable.

The app wasn't malicious, and it wasn't abandoned out of spite. It was simply another example of a reality every developer eventually faces: maintaining software indefinitely is hard.

What surprised me wasn't that the application was deteriorating. What surprised me was how difficult it was to leave.

The Data Extraction Nightmare

I assumed exporting my data would be straightforward. It wasn't.

The app no longer provided a working export mechanism, and because my phone wasn't rooted, I couldn't simply pull the underlying database file over ADB. I was trapped in a walled garden of an app that didn't even have a wall. Just a broken gate.

So, I ended up writing a Python script that used Android Debug Bridge (ADB) and uiautomator2 to semi-automatically click through every single screen of the application, parsing the view tree and scraping my own reading history out of the UI elements.

Let that sink in: To recover data I had manually entered myself over several years, I had to screen-scrape my own phone.

That experience changed how I think about personal data. If exporting your information requires reverse engineering, UI automation, or screen scraping, you don't actually own it.

Looking for Alternatives

Naturally, I started evaluating alternatives. I checked out commercial platforms, SaaS products, and existing self-hosted projects. Each solved part of the problem, but none solved all of it.

I wanted a very specific set of constraints:

Absolute data ownership: A storage format I can drop into a backup script.
Offline-friendly core: If my server loses internet, the UI should still render my existing books instantly.
Rich analytics: Reading heatmaps, monthly progress statistics, author statistics and more.
A documented API: Headless access so I can pipe data into home automation, dashboards and other tools of my self-hosted stack.
Lightweight footprint: Simple enough to run on a low-powered device. (No bloated Java stack that takes minutes to boot...)

Many solutions came close, but every candidate had a compromise I wasn't willing to make. Some queried Open Library live on every single page render (meaning no internet = no library view). Others introduced subscriptions or heavy vendor lock-in.

Eventually, I reached the conclusion many developers know well:

Building my own solution was less frustrating than continuing a compromised search.

Designing LibrisLog

The result is LibrisLog: an open-source, multi-user, self-hosted reading tracker focused entirely on data portability.

One core design principle influenced almost every architectural decision: External services should improve the experience, but they must never become a single point of failure.
If you use the built-in mobile browser barcode scanner to look up an ISBN, LibrisLog hits Open Library, Google Books, or Hardcover.app to fetch metadata and cover art. But once that data hits your local instance, it is normalized, stored locally, and entirely yours. If those external APIs disappear tomorrow, your library remains fully intact, completely readable, and fully editable.

A Deliberately Boring Stack

The technology choices for the project were intentionally conservative. I didn’t want to spend my weekends maintaining the infrastructure of my tracking app.

Backend: FastAPI, SQLModel, SQLite, Alembic
Frontend: Svelte 5, SvelteKit, Tailwind CSS v4, DaisyUI v5
Deployment: Docker Compose with separate backend and frontend containers

No Kubernetes. No distributed databases. No microservices. Just two containers and a single SQLite file.

The goal wasn't maximum web-scale scalability; it was ease of self-hosting. By treating the backend as a completely headless REST API, the application automatically gains a fully documented Swagger UI (/api/docs). Users can generate API keys directly from the web UI, making it incredibly simple to write scripts or display reading stats on personal dashboards like Dashy, Homepage, or Home Assistant.

Engineering for Messy Data

When you build an app designed to help people escape proprietary platforms, you quickly realize that legacy exports are a disaster.

Goodreads gives you a CSV, but some other apps give you highly non-standard JSON formats or weirdly formatted spreadsheets. To solve this without hardcoding an infinite number of migration scripts, I built a flexible import engine that supports custom per-field transforms using inline Python expressions.

If a legacy app exported your reading dates as a non-standard timestamp string or a localized date format, you can define a quick Python transform snippet directly during the import phase. The backend executes these safely to normalize the incoming data on the fly before it ever hits the SQLite database.

Accelerating Solo Development with a Spec-Driven Workflow

Building a full-stack, multi-user application as a side project means managing tight time constraints. To compress the timeline from a blank terminal to a functional Docker image, I integrated LLMs (Opencode) directly into my development loop to handle boilerplate and test generation.

The workflow was highly decoupled and spec-driven:

Architecture First: I designed the database models, API contracts, and frontend state management strategies.
Explicit Blueprints: I provided the AI tool with strict markdown execution plans detailing exactly how a utility or endpoint should behave.
Parallel Test Generation: The AI generated the implementation code alongside a corresponding suite of pytest or Vitest unit tests based on the spec.
Manual Review: Every line of generated code went through a manual review and refactoring pass before hitting main.

Using the tool this way eliminated the friction of repetitive UI styling, routine endpoint framing, and test setup. It didn't replace architectural decisions, but it acted as a massive force multiplier for execution speed.

Unexpected Lessons

The biggest takeaway from this project wasn't technical. It was realizing how much software longevity depends on data portability.

Features come and go. Interfaces change. Projects are abandoned. Maintainers move on (I recently passed the torch on a 10-year-old open-source project of my own, so I know how it goes). But users want access to the data they've accumulated over a lifetime.

Building software that respects that reality feels increasingly important in an era where so much modern software assumes a permanent connection to a corporate cloud server that might not exist tomorrow.

Ironically, the most valuable feature I built into LibrisLog wasn't the statistics dashboard, the OIDC authentication, or the barcode scanner.

It was the ability to leave. LibrisLog lets you export everything as ZIP file containing all data and your covers with a single click. Software should make it easy for users to stay, but it must make it easy for them to leave. And when users know they can walk away with their data at any second, they're much more likely to stay.

If you've been looking for a lightweight, self-hosted way to own your reading history, feel free to check out the project, open an issue, or contribute: