DEV Community: Ankan Roy

Ghost in the Machine: Privacy-First Local RAG Microservice with FastAPI

Ankan Roy — Tue, 27 Oct 2026 07:19:33 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

🌟 The Motivation: Why Open-Source AI Matters

Cloud-based AI APIs have simplified adding intelligence to applications, but they come with significant trade-offs: network latency, recurring subscription costs, rate limits, and vendor lock-in. More importantly, sending sensitive codebase or enterprise documentation to third-party endpoints creates serious data privacy concerns.

For Week 4's "Ghost in the Machine" challenge, I wanted to build a zero-cloud-dependency microservice that runs vector indexing and retrieval completely locally.

By keeping vector embedding and context matching on your local machine or private server, you retain 100% control over your data while enjoying sub-millisecond retrieval speeds.

🚀 What I Built: GhostInTheMachine RAG

GhostInTheMachine RAG is an asynchronous Retrieval-Augmented Generation (RAG) microservice built with FastAPI and Python 3.13.

Instead of pulling heavy external LLM packages, it features an in-memory term-frequency vector embedding engine with cosine similarity indexing built from scratch. It allows developers to ingest internal knowledge documents, store them securely in local memory, and query relevant contexts with sub-millisecond latency.

🛠 Tech Stack & Architecture

Backend Engine: FastAPI (Asynchronous REST framework)
Runtime & Server: Python 3.13 + Uvicorn ASGI Server
Data Validation: Pydantic v2
Vector Mechanics: In-Memory Term Frequency Vector Embedding + Cosine Similarity Scoring
Developer Workflow: Visual Studio Code & DevRelay MCP Gateway Integration

💡 How It Works

The microservice processes incoming knowledge requests through a lean, three-stage local pipeline:

Document Ingestion (POST /ingest): Text content is tokenized, frequency-normalized into term vectors, and stored in the local vector store alongside metadata.
Vector Similarity Search (POST /query): Inbound queries are converted into normalized term vectors on the fly. The engine computes cosine similarity against indexed documents to retrieve top-$k$ matches.
Context Synthesis: Retrieved contexts are aggregated and synthesized into an immediate, zero-latency response payload without making external API calls.

💻 Core Implementation Highlights

Here is how the core local vector similarity engine is structured in app/services/rag_engine.py:


python
import time
import math
from typing import List, Dict

class SimpleVectorStore:
    def __init__(self):
        self.documents: List[Dict] = []

    def _tokenize(self, text: str) -> List[str]:
        return [w.lower() for w in text.split() if w.isalnum()]

    def _embed(self, text: str) -> Dict[str, float]:
        tokens = self._tokenize(text)
        if not tokens:
            return {}
        freq = {}
        for t in tokens:
            freq[t] = freq.get(t, 0) + 1.0
        norm = math.sqrt(sum(v ** 2 for v in freq.values()))
        return {k: v / norm for k, v in freq.items()}

    def _cosine_similarity(self, vec1: Dict[str, float], vec2: Dict[str, float]) -> float:
        score = 0.0
        for k, v in vec1.items():
            if k in vec2:
                score += v * vec2[k]
        return score

    def search(self, query: str, top_k: int = 3) -> List[Dict]:
        q_vec = self._embed(query)
        scored = []
        for doc in self.documents:
            sim = self._cosine_similarity(q_vec, doc["vector"])
            scored.append((sim, doc))
        scored.sort(key=lambda x: x[0], reverse=True)
        return [doc for sim, doc in scored[:top_k] if sim > 0]

Building DevPulse: An Automated Code Health & Analytics Engine with FastAPI

Ankan Roy — Tue, 20 Oct 2026 04:48:58 +0000

# Building DevPulse: An Automated Code Health & Analytics Engine

🛠️ Hacktoberfest 2026 Week 3 Submission: Developer Tooling & Automation

For Week 3 of the Hacktoberfest 2026 DEV Challenge, I built DevPulse—an automated developer utility API using FastAPI that evaluates code snippets, calculates complexity metrics, and generates instant architectural feedback.

🚀 What I Built

An asynchronous FastAPI microservice designed to inspect code structures, evaluate cyclomatic complexity, and provide clean maintainability ratings for backend repositories.

AnkanJU / Hacktoberfest2026

Monorepo for Hacktoberfest 2026 projects, DEV.to AI Challenges, and Open-Source backend implementations.

View on GitHub

Tech Stack: Python 3.13, FastAPI, Uvicorn, Pydantic.
Theme Alignment: Developer Tools & Workflow Automation.

💡 Why Developer Tooling Matters

Automating code health checks helps engineering teams catch structural bottlenecks early, ensuring robust, high-performance deployments across microservices.

🏃 How to Run Locally


bash
# Clone repository
git clone [https://github.com/AnkanJU/Hacktoberfest2026.git](https://github.com/AnkanJU/Hacktoberfest2026.git)
cd Hacktoberfest2026/dev-challenge-week4-tools

# Install dependencies
pip install -r requirements.txt

# Run server
uvicorn main:app --reload
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/i3b5gpg024jtlo3h7ikp.png)
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/y2jjo9cdit0cfua0aca1.png)

Building NexusSaaS: A Monetized AI Micro-API Gateway with FastAPI

Ankan Roy — Tue, 13 Oct 2026 13:02:33 +0000

# Building NexusSaaS: A Monetized AI Micro-API Gateway

💼 Hacktoberfest 2026 Week 2 Submission: Commercial AI / Side Hustle

For Week 2 of the Hacktoberfest 2026 DEV Challenge, I built NexusSaaS—a lightweight, monetization-ready micro-API gateway using FastAPI that validates API subscription keys and tracks token usage for a developer-focused tool.

🚀 What I Built

An asynchronous FastAPI backend featuring API key authentication headers, tiered access control, and usage metering designed to be packaged as a commercial developer tool.

AnkanJU / Hacktoberfest2026

Monorepo for Hacktoberfest 2026 projects, DEV.to AI Challenges, and Open-Source backend implementations.

View on GitHub

Tech Stack: Python 3.13, FastAPI, Uvicorn, Pydantic.
Theme Alignment: Side Hustle / Monetizable Developer API.

💡 Why Micro-SaaS APIs Matter

Building modular, API-first microservices allows developers to easily prototype commercial products, test subscription mechanics, and manage secure header authentication locally before deploying to cloud infrastructure.

🏃 How to Run Locally


bash
# Clone repository
git clone [https://github.com/AnkanJU/Hacktoberfest2026.git](https://github.com/AnkanJU/Hacktoberfest2026.git)
cd Hacktoberfest2026/dev-challenge-week3-saas

# Install dependencies
pip install -r requirements.txt

# Run server
uvicorn main:app --reload

TrailWise AI: Local Outdoor Trail & Activity Advisor (#TouchGrass)

Ankan Roy — Wed, 07 Oct 2026 16:49:49 +0000

# TrailWise AI: Local Outdoor Trail & Activity Advisor

🎃 Hacktoberfest 2026 Week 1 Submission: Touch Grass 🌿

For Week 1 of the Hacktoberfest 2026 DEV Challenge ("Touch Grass"), I built TrailWise AI—a lightweight, local AI assistant designed to give users quick gear checklists, route advice, and trail safety tips so they can minimize screen time and spend more time outdoors.

🚀 What I Built

An asynchronous FastAPI engine running open-weight local inference (llama3.2 via Ollama). The backend accepts outdoor activity parameters and instantly generates actionable trail and safety advice designed to work completely offline.

AnkanJU / Hacktoberfest2026

Monorepo for Hacktoberfest 2026 projects, DEV.to AI Challenges, and Open-Source backend implementations.

View on GitHub

Tech Stack: Python 3.13, FastAPI, Ollama, Docker.
Theme Alignment: Touch Grass (Trail safety, outdoor preparation, offline-first access).

💡 Why Open-Source AI Matters

Outdoor activities often take place in areas with zero cellular connectivity. Running llama3.2 locally allows users to generate trail safety plans and gear checklists anywhere on a laptop or local edge node without relying on third-party cloud APIs.

🛠️ API Interface Example


json
// POST /advise
{
  "activity": "hiking",
  "location_or_climate": "temperate fall foliage"
}

Building a Resilient Local RAG Backend Engine for Hacktoberfest 2026

Ankan Roy — Sun, 04 Oct 2026 15:05:39 +0000

# Building a Resilient Local RAG Backend Engine

🎃 Hacktoberfest 2026 Submission

For the Hacktoberfest 2026 DEV Challenge, I built a zero-dependency, local-first Retrieval-Augmented Generation (RAG) backend engine designed to run open-weight models completely offline.

🚀 What I Built

An asynchronous FastAPI backend paired with PostgreSQL (pgvector) and Ollama (llama3.2). The system allows querying local AI models with vector context retrieval without sending sensitive data to third-party cloud APIs.

AnkanJU / Hacktoberfest2026

Monorepo for Hacktoberfest 2026 projects, DEV.to AI Challenges, and Open-Source backend implementations.

View on GitHub

Tech Stack: Python 3.13, FastAPI, Uvicorn, AsyncPG, Docker Compose, Ollama.

💡 Why Open-Source AI Matters

Running open-weight models like llama3.2 locally gives developers full data ownership and privacy. By keeping embeddings in pgvector and processing prompts locally via Ollama, sensitive data never leaves the local environment.

🛠️ Key Features & Architecture

Asynchronous API: Built using FastAPI for low-latency request handling.
Local Vector Search: PostgreSQL with the pgvector extension configured for HNSW indexing.
Local Inference: Containerized Ollama instance running open-weight LLMs.
Resilient Retries: Integrated fallback handling for reliable local execution.

🏃 How to Run Locally


bash
# Clone the repository
git clone [https://github.com/AnkanJU/Hacktoberfest2026.git](https://github.com/AnkanJU/Hacktoberfest2026.git)
cd Hacktoberfest2026/dev-challenge-week1-rag

# Install dependencies
pip install -r requirements.txt

# Start local infrastructure
docker compose up -d

# Run the API server
uvicorn main:app --reload
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/y7g8jqwwapir8k498m3n.png)
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/oqjise1tojhjj3uqx5h7.png)

Building a Resilient Local RAG Backend Engine for Hacktoberfest 2026

Ankan Roy — Sun, 04 Oct 2026 14:53:47 +0000

# Building a Resilient Local RAG Backend Engine

🎃 Hacktoberfest 2026 Submission

For the Hacktoberfest 2026 DEV Challenge, I built a zero-dependency, local-first Retrieval-Augmented Generation (RAG) backend engine designed to run open-weight models completely offline.

🚀 What I Built

GitHub Repository: https://github.com/AnkanJU/Hacktoberfest2026
Tech Stack: Python 3.13, FastAPI, Uvicorn, AsyncPG, Docker Compose, Ollama.

💡 Why Open-Source AI Matters

Running open-weight models like llama3.2 locally gives full data ownership and privacy. By keeping embeddings in pgvector and processing prompts locally via Ollama, sensitive data never leaves the developer's workstation.

🛠️ Key Features & Architecture

Asynchronous API: Built using FastAPI for low-latency request handling.
Local Vector Search: PostgreSQL with the pgvector extension configured for HNSW indexing.
Local Inference: Containerized Ollama instance running open-weight LLMs.
Resilient Retries: Integrated error handling and retries for reliable local processing.

🏃 How to Run Locally


bash
# Clone the repository
git clone [https://github.com/AnkanJU/Hacktoberfest2026.git](https://github.com/AnkanJU/Hacktoberfest2026.git)
cd Hacktoberfest2026/dev-challenge-week1-rag

# Install dependencies
pip install -r requirements.txt

# Start local infrastructure
docker compose up -d

# Run the API server
uvicorn main:app --reload
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/bgvrtauy2y7zf4ffsalv.png)
![ ](https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/m4wqgxbg2evb2aikg0z0.png)