Building a "chat with your PDF" tool is a common entry point into AI, but the real challenge lies in making it reliable under hardware constraints. I’ve spent the last few months developing Research-OS (R-OS), a retrieval-augmented generation (RAG) system designed to handle complex academic papers locally on a 16GB RAM machine.
In this first post, I want to share the architectural decisions I made after a significant environment failure taught me the importance of dependency isolation and resource-efficient modeling.
The Core Tech Stack:
Vector Engine: FAISS for efficient similarity search.
Embeddings: FastEmbed (chosen for its low memory footprint compared to standard transformers).
Orchestration: FastAPI for a decoupled backend-frontend communication.
Logic: A hybrid retrieval system combining BM25 and vector-based search.
R-OS isn't just about calling an LLM API; it’s about the pipeline—how we ingest data, manage chunks, and verify that the retrieved context actually answers the user's query without hallucinating.
Github Link : https://github.com/sharansh-22/Research-OS
Top comments (0)