My boy reads dozens of light novels and manga every month. He let one of his Novels to marinate too long and came back completely lost—he'd forgotten who the main villain was and why the protagonist was even fighting them!😂
I built Page-Grindr to help him, since stuff like this is quite common. It's a Local AI helper that helps you backtrack so you can remember where you're at, you just have to upload your epub or pdf file.
It's like having a personal fandom wiki for every book you read, powered by tinyllama that runs 100% locally on your laptop.
Demo
https://runway-fled-hypnosis.ngrok-free.dev
The app runs locally, but I used Ngrok to create a secure tunnel so my friend could test it remotely while keeping all data on my machine.
Code Repo
https://github.com/HeroOfMyLife/Page-Grindr
Honestly, this was a 2-day rush and I hit a few walls. Like I kept forgetting to add my Mongodb password to app.py.
I started out wanting to use Google's Gemma model, but my laptop's RAM completely choked on it. I had to pivot to TinyLlama running locally through Ollama. It’s much smaller, but for searching through text chunks, it works perfectly with some small hiccups here and there.
For the "memory," I used MongoDB Atlas. I wrote a JSON script that takes the uploaded book, chops it into 1000-character chunks, turns those chunks into numbers (embeddings) using an open-source model called nomic-embed-text, and saves them to the database.
When my friend asks a question, I turn his question into numbers too, ask MongoDB to find the 3 most similar chunks of text, and feed those to TinyLlama to generate the answer.
I also added a sidebar where she can upload the books herself, and a button that uses ElevenLabs to read the AI's answer out loud like an audiobook, although I doubt he'll use it since he's not really an audiobook guy.
There are two big reasons I did this with open-source tools instead of just hooking it up to ChatGPT:
Privacy: Light novel readers can be weirdly protective of their reading lists. By running the AI locally with Ollama, his books never get uploaded to a third-party server. Plus I'm an up and coming privacy nut, can't trust big tech too much these days.
Cost: If I used a paid API, processing a 25MB light novel would cost actual money every time my dude uploaded a new book. With open-source models, it costs $0 to run. And honestly I don't have the cash for that currently, I normally just support them when I get pocket money.😅
It also meant I could swap models on the fly when my computer ran out of memory. If I was locked into a closed API, the project would have just died when Gemma crashed my RAM.
Prize Categories
I'm going for
Best Use of MongoDB Atlas: Used Vector Search to handle the massive amount of text from the books.
Best Use of ElevenLabs: Used it to generate audio recaps of the lore.
This is my First Post So I hope I can get some good feed back. And thank you to Dev community and MLH I hope this Octoberfest goes really well.
Top comments (0)