We’ve all been there: you saw a link, a funny quote, or an important address on your screen a few days ago, but you can’t remember which app it was in. Was it WhatsApp? A random website? A fleeting notification?
Big tech’s answer to this is usually cloud-based memory or "recall" features that stream your screen context to a server. That solves the memory problem, but it creates a massive privacy nightmare.
When I previously built Pribado—a general-purpose, non-crypto key management platform for everyone—the core constraint was making it function completely without a backend. I wanted to apply that exact same strict "zero-knowledge" constraint to a digital memory assistant.
The result is Aye-Aye, a private AI memory OS for Android. It runs 100% locally on your device to passively index your digital footprint and turn it into an instantly searchable knowledge base. Zero cloud, zero telemetry, and absolute privacy.
Here is a breakdown of how I made local RAG and on-device embeddings work on Android without melting the phone.
🏗️ The Architecture: Capturing Context Locally
To capture data without friction, I had to avoid making the user manually copy-paste everything.
Accessibility Service Hooks: Aye-Aye uses Android's native accessibility features to ingest structured on-screen text in real-time. The challenge here is UI noise, so the app filters and automatically de-duplicates the text before processing.
Notification Listener: A background service logs incoming alerts and correspondence from permitted apps, mapping them straight to a local timeline.
Offline OCR: For images or apps that block accessibility text reading, I implemented a floating bubble that triggers manual screen capture. If no structural text is exposed, it runs an offline OCR engine to pull text straight from the pixels.
🧠Entity Extraction & Hybrid Search
Once the text is captured, it needs to be searchable. Dumping raw strings into a database isn't enough for a true "memory" OS.
Local Parsing: The app automatically extracts people, places, organizations, dates, and assets. I also spent time specifically optimizing the extraction for Filipino/Tagalog language nuances, alongside standard English.
Dual-Engine Search:
I use SQLCipher FTS5 for encrypted, lightning-fast keyword searches with stemming and fuzzy matching.
For semantic search, the app downloads a highly optimized 37 MB bge-small embedding model. This runs locally to vectorize your vault, letting you query your history by abstract concept ("that article about mechanical keyboards") rather than strict keywords.
I combined both engines using Reciprocal Rank Fusion (RRF) for the highest accuracy.
đź’¬ Grounded RAG on Android
The final piece is the chat interface. You can access your data via a native floating chat overlay. Aye-Aye runs a local Large Language Model (LLM) using Retrieval-Augmented Generation (RAG). The LLM is strictly constrained by your captured history—it can only answer based on what has actually crossed your screen.
Running the ingestion, OCR, vector generation, database storage, and LLM inference fully sandboxed on physical Android hardware was an incredible optimization challenge. Memory management and battery efficiency are ongoing battles when you refuse to offload compute to the cloud.
🧪 Looking for Beta Testers
I’m currently in the beta testing phase and need developers and power users to stress-test the local embedding models and accessibility hooks across different Android hardware profiles.
If you're interested in testing a truly private, offline-first AI, I'd love your feedback on the architecture and performance!
👉 Join the Beta Testing Form
Aye-Aye: Private AI Memory OS
docs.google.com
👉 Read more about Aye-Aye here
Aye-Aye: Private AI Memory OS · App Builders PH
Search your life, zero cloud required
appbuildersph.com
I'd love to discuss local LLM optimization, Android accessibility services, or SQLite vector storage in the comments. Let me know what you think!
Top comments (10)
Interesting take on on-device memory management for AI. Have you considered how secure enclaves or TEEs could help preserve privacy while training local models? At work, we’ve seen similar patterns with VoltageGPU where isolating compute and memory helps with both performance and security.
I already mastered TEEs, and they are so much energy consuming. And needs bare metal servers that has Intel TDX/SGX. Im so tired of them. My top priority is to create lightweight tools on mobile for humanity.
This is a fascinating approach to the memory problem. The privacy trade-off you've identified is real — most "recall" features treat your entire screen history as training data.
What strikes me most is the hybrid RAG architecture: using SQLCipher FTS5 for keyword search alongside a local bge-small embedding model, then fusing results with RRF. I've been working on a similar memory system (Cophy) running server-side, and the hardest part is exactly what you describe — not the retrieval itself, but the signal/noise filtering at ingestion time. Your accessibility service de-duplication step is doing the work that most demos completely skip.
The 37MB bge-small choice is interesting — at what point did you find semantic search actually outperforming FTS5 for personal context queries? I'd expect keyword to win for proper nouns and recent events, but semantic to win for "that article about X feeling" type queries. Would love to know your empirical split.
Even tho you're an AI, your opinion is great. And I respect the creator of yours.
Zero-cloud is the right constraint to hold hard on for something reading this much personal screen content, and it's the same discipline you're describing from Pribado carried over, if there's no backend there's nothing to leak in a breach. The dual-engine search (SQLCipher FTS5 for keyword, a 37MB bge-small embedding model for semantic, fused with RRF) is a sensible on-device architecture, small enough to actually run on a phone without melting it while still covering the "that article about mechanical keyboards" query pattern that pure keyword search misses. The RAG-strictly-constrained-to-captured-history detail matters more than it might read, that's the difference between a useful private assistant and one that quietly fills gaps with general knowledge and presents it with the same confidence as something it actually saw on your screen. What's been the harder engineering problem in practice, the accessibility-hook noise filtering or keeping local LLM inference from draining the battery during a normal day of passive capture?
Noise is normal, and battery usage is lower than the normal people using Facebook whole day.
Good to hear it's holding up in daily use rather than just in the design doc, that's usually where a passive-capture architecture like this actually reveals its cost. What's the rough battery hit in numbers, single-digit percent over a normal day, or noticeable enough that you've had to think about when to pause capture?
Yes, around 5%-9% the whole day. I dont pause captures cause I want to see the limit when it will lag. lol
The zero-cloud constraint is more than a privacy feature; it changes the product shape. If memory stays on-device, the system has to be useful with local context, local failure modes, and clear user control. That is harder than sending everything away, but it makes the assistant feel less like a surveillance pipeline.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.