Semantic Search Without an LLM. Build Blazing-Fast Semantic Search for \$0

UMER ABDULLAH — Wed, 23 Sep 2026 08:29:10 +0000

At the moment, we are in the period of "AI search" which is the use of effective and slow Generative LLMs to answer basic queries. But we are approaching the "generative AI fatigue" turning point.

According to the global ecosystem research, there are more than 80% of programmers addicted to generic AI assistants. Constant cravings for confirmation led to the sharp decline of the trust in generative effectiveness.

Even major corporations have limited the use of generative technology. This is evident in the internal policies of the companies like Amazon which make their engineers avoid using any third-party sources.

The explanation of this situation can be traced back to the Pareto principle in Product Design. The research conducted by Pendo has demonstrated that 80% of the average product functionality is connected to the basic search process.

User Text Data ──> Hugging Face Inference API ──> Dense Vectors (384 Dimensions)
                                                            │
                                                            ▼
GitHub Pages Frontend <── [Sub-Millisecond Search] <── Qdrant In-Memory Cluster

1. The Production Backend Server (`app.py`)

This FastAPI script sets up a temporary Qdrant database and monitors incoming payloads actively. It retrieves access tokens from the environment and provides the routes needed for the bulk upload of documents (/upload) and for searching for vector matches (/search).

import os
import json
import requests
from fastapi import FastAPI, UploadFile, File
from fastapi.middleware.cors import CORSMiddleware
from qdrant_client import QdrantClient, models

app = FastAPI()

# Secure Cross-Origin Resource Sharing to accept external web traffic smoothly
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Safely loading the Hugging Face Token from environment variables
HF_TOKEN = os.getenv("HF_TOKEN")
API_URL = "https://huggingface.co"
headers = {"Authorization": f"Bearer {HF_TOKEN}"}

def get_embeddings(texts: list):
    response = requests.post(API_URL, headers=headers, json={"inputs": texts})
    return response.json()

# Initializing an un-mapped, highly isolated Qdrant vector collection inside system RAM
client = QdrantClient(location=":memory:")
COLLECTION_NAME = "enterprise_docs"

client.create_collection(
    collection_name=COLLECTION_NAME,
    vectors_config=models.VectorParams(size=384, distance=models.Distance.COSINE)
)

@app.post("/upload")
async def upload_data(file: UploadFile = File(...)):
    try:
        contents = await file.read()
        data = json.loads(contents.decode("utf-8"))

        documents = [item["text"] for item in data]
        metadata_payload = [{"category": item.get("category", "general"), "document": item["text"]} for item in data]
        generated_ids = list(range(1, len(documents) + 1))

        vector_lists = get_embeddings(documents)

        if isinstance(vector_lists, dict) and "error" in vector_lists:
            return {"status": "error", "message": f"HF API Error: {vector_lists['error']}"}

        client.upload_collection(
            collection_name=COLLECTION_NAME,
            vectors=vector_lists,
            payload=metadata_payload,
            ids=generated_ids
        )
        return {"status": "success", "message": f"Successfully indexed {len(documents)} chunks!"}
    except Exception as e:
        return {"status": "error", "message": str(e)}

@app.get("/search")
async def search_data(query: str, category: str = "All Categories"):
    query_filter = None
    if category and category != "All Categories":
        query_filter = models.Filter(
            must=[models.FieldCondition(key="category", match=models.MatchValue(value=category))]
        )

    try:
        query_vector = get_embeddings([query])

        search_result = client.query_points(
            collection_name=COLLECTION_NAME,
            query=query_vector,
            query_filter=query_filter,
            limit=3
        ).points

        results = []
        for hit in search_result:
            results.append({
                "score": round(hit.score, 4),
                "category": hit.payload.get("category", "general"),
                "text": hit.payload.get("document", "")
            })
        return {"status": "success", "results": results}
    except Exception as e:
        return {"status": "error", "message": str(e)}

@app.get("/")
def home():
    return {"message": "Zero-LLM Search API is Live!"}

2. Server Dependencies (`requirements.txt`)

To launch this service, keep your dependencies as lean as possible:

fastapi
uvicorn
python-multipart
qdrant-client
requests

3. The Test Dataset Matrix (`knowledge_base.json`)

To check the vector grouping capability, you can load this structured data template snippet directly into the interface:

[
  {
    "text": "To secure your API endpoints, always configure JWT authentication middleware to intercept incoming requests and validate the token signature.",
    "category": "security"
  },
  {
    "text": "OAuth2 authorization code flows with PKCE are highly recommended for single-page applications and mobile apps to securely exchange tokens.",
    "category": "security"
  },
  {
    "text": "For low-latency API verification, store client API keys in a Redis caching backend with a 5-minute TTL to drastically reduce primary database strain.",
    "category": "devops"
  },
  {
    "text": "When deploying code, configure an automated CI/CD pipeline inside GitHub Actions to trigger unit testing with the Jest framework on every pull request.",
    "category": "testing"
  }
]

Why This Structure Works

When using the application structure in this manner, questions like "how do I safeguard my routing configurations?" can be achieved with just a few clicks. The solution interprets context and meaning but makes use of the AI algorithms in a reasonable way.

Cost-Free: The overall price is $0 per month because the service providers are free to use any service from GitHub Pages or Render. The variable tokens are also free of charge.
No Waiting Time: The dot-product cosine matching timings are done in RAM in less than 10 milliseconds, whereas AI takes about 2-5 seconds to create an answer.
No 'Hallucinations': There is no chance of users receiving 'hallucinations' from the machine. It only presents lines of code written by developers without making any assumptions.

⚠️ Important Details on the Infrastructure: Free Plan Drawbacks

The application functions on public cloud services, and thus the provider puts the Python engine to sleep mode when no activity happens within a period of time.
When users try to send a request after using the application for some period, it takes the server around 30-60 seconds to wake up.

Project Links and Resources

🛠️ Full Source Code Repository: GitHub - Zero-LLM-search
🚀 Live Interactive Demo Page: Launch Web Engine Site

Drop a star on the repo if you found this template helpful, and let me know in the comments how you plan to use serverless semantic search in your own apps!

DEV Community: UMER ABDULLAH