DEV Community

Cover image for Building NovaDesk AI: Moving Beyond Basic RAG with Query Rewriting and HyDE
Ananya S
Ananya S

Posted on Edited on AI-assisted

Building NovaDesk AI: Moving Beyond Basic RAG with Query Rewriting and HyDE

Setting up a basic RAG (Retrieval-Augmented Generation) pipeline takes ten minutes. You load a few PDFs, chunk them, push the embeddings into a vector store like FAISS, and run a similarity search.

In a clean demo, it looks like magic. In practice, it breaks down the moment a real user types a messy, conversational question.

While building NovaDesk AI—my stateful hotel concierge powered by LangGraph, FastAPI, and SQLite—I needed the system to answer unstructured hotel policy questions reliably without hallucinating. This meant moving past naive retrieval.

You can check out the previous post here where I have introduced this project:

In this post, I will walk through how I evolved the RAG pipeline in NovaDesk AI across three distinct stages: Standard RAG, Query Rewriting, and HyDE (Hypothetical Document Embeddings).


Stage 1: Standard (Naive) RAG and Where It Broke

My initial setup was a textbook RAG pipeline. I took the hotel policy documents (covering check-in times, pet rules, cancellation fees, and amenities), split them into chunks, and indexed them in a FAISS vector store.

When a user asked a question, the system embedded the raw query, fetched the top-k most similar chunks, and passed them to the LLM to generate an answer.

The Problem

Standard RAG works well when the user's phrasing closely mirrors the wording in your documents (e.g., "What is the standard check-in time?").

However, real hotel guests do not speak like policy manuals. When I tested the concierge with realistic, messy queries, retrieval quality dropped sharply:

  • Laymen language & typos: "Can I bring my husky and how much extra?"
  • Multi-part questions: "Do you have airport pickup and is breakfast free if I book a deluxe room?"
  • Conversational noise: "Hey, I'm running super late for my flight, will I lose my room if I show up at 2 AM?"

Because the raw user query was full of conversational filler—and structurally very different from formal policy text—the vector search frequently pulled irrelevant chunks or missed half of a multi-part question.


Stage 2: Introducing a Query Rewriter

Before changing the vector database or tweaking chunk sizes, I looked at the input. Garbage in, garbage out.

To fix this, I added a Query Rewriter step right before the retrieval call. Instead of embedding the user's raw message, an LLM first translates the messy input into a clean, search-optimized query.

  • Raw User Query: "Hey, can I bring my husky with me and how much extra will it cost?"
  • Rewritten Query: "Hotel pet policy, allowed dog breeds, pet fees and charges"

Here is a simplified look at how this step is structured using LangChain:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# =====================================================================
# 2. QUERY TRANSFORMATION CHAINS (HYDE & QUERY REWRITER)
# =====================================================================
# Mode A: Query Rewriter Prompt (Strips conversational noise into keywords)
rewriter_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an expert search query optimizer for a hotel policy database. "
               "Rewrite the guest's message into a short, direct, keyword-focused search query. "
               "Strip out all personal backstory, names, and conversational filler. "
               "If the question has multiple parts, include keywords for all parts."),
    ("human", "{user_query}")
])
rewriter_chain = rewriter_prompt | llm | StrOutputParser()


# Usage before hitting FAISS
optimized_query = rewriter_chain.invoke({"user_query": user_input})
docs = retriever.invoke(optimized_query)
Enter fullscreen mode Exit fullscreen mode

The Impact

The Query Rewriter immediately stripped away conversational fluff and standardized the vocabulary. Multi-part questions and casual phrasing started hitting the right sections of the policy documents far more consistently.

Stage 3: Bridging the Semantic Gap with HyDE

Even with a Query Rewriter, there is a fundamental flaw in standard vector search: questions and answers do not look alike in vector space.

A user's query is short and interrogative ("What happens if I show up at 2 AM?"). The policy document is long and declarative ("Guests arriving after 11:00 PM must notify the front desk in advance to prevent a no-show cancellation..."). Comparing a question embedding against a document embedding is an asymmetric match.

To solve this, I implemented HyDE (Hypothetical Document Embeddings).

How HyDE Works

Take the user's question (or rewritten query).

Ask the LLM to generate a hypothetical, ideal policy excerpt that would answer the question. (It does not matter if the LLM hallucinates specific numbers here—we are only using it for semantic structure).

Embed that hypothetical document and use it to search the FAISS vector store.

Pass the real retrieved policy chunks to the final LLM to generate the factual answer.

# Mode A: HyDE Prompt (Generates a formal hypothetical policy paragraph)
hyde_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an expert at writing formal hotel policy documents. "
               "The user will ask a question. Write a short, single-paragraph hypothetical hotel policy "
               "that perfectly answers their question. Use professional, legal hospitality terminology."),
    ("human", "{user_query}")
])
hyde_chain = hyde_prompt | llm | StrOutputParser()

# 1. Generate the hypothetical answer
hypothetical_doc = hyde_chain.invoke({"user_query": optimized_query})

# 2. Search FAISS using the hypothetical document instead of the raw question
retrieved_docs = retriever.invoke(hypothetical_doc)

Enter fullscreen mode Exit fullscreen mode

The Impact & Trade-offs

HyDE shifted the search from question-to-document matching to document-to-document matching. For nuanced, indirect questions where the user did not know the exact terminology (like "no-show policy" or "late arrival"), HyDE pulled significantly more relevant context than standard retrieval.

The Trade-off: Latency and token cost. Adding a Query Rewriter and a HyDE step introduces extra LLM calls before you even generate the final answer. In a real-time concierge, you have to balance retrieval accuracy against how long the user is willing to wait for a response.

Key Takeaways

Fix retrieval before upgrading your LLM: When a RAG system hallucinates or gives incomplete answers, the bottleneck is almost always the context being fed into the prompt, not the reasoning power of the model.

Users do not write like your database: Adding a lightweight Query Rewriter is one of the highest-ROI improvements you can make to a conversational RAG pipeline.

HyDE shines on indirect queries: When there is a wide vocabulary gap between how users ask questions and how your documentation is written, generating a hypothetical answer bridges that gap effectively.

You can explore the full codebase for NovaDesk AI on my GitHub repository.

🛎️ AI Hotel Concierge (LangGraph + FastAPI)

An enterprise-grade, stateful AI agent built to simulate a virtual hotel front desk. Unlike basic Q&A chatbots, this system bridges the gap between unstructured natural language and strict database transactions, allowing users to query policies, check room availability, book rooms, and request room service seamlessly.

This project demonstrates advanced AI engineering concepts including Agentic Orchestration (LangGraph), Advanced RAG (HyDE & Query Rewriting), State & Memory Management, and Human-in-the-Loop (HITL) security via FastAPI.

🏗️ Architecture Overview

The system is broken down into three main layers:

Frontend (Chainlit): Provides a clean, conversational chat UI.

Orchestration Brain (LangGraph): Manages conversation state, memory, guardrails, and decision routing.

Action Execution (FastAPI): Serves as the toolkit for the agent, exposing REST endpoints to interact securely with the SQLite database.

System Architecture

Agent Architecture

✨ Key Features

  • Strict Domain Guardrails: A routing node intercepts prompts. If a user asks off-topic questions (e.g…





Have you implemented HyDE or Query Rewriting in your RAG pipelines? Did you find the accuracy boost worth the extra LLM latency? Let me know below! 👇

Top comments (3)

Collapse
 
zeroshotanu profile image
Ananya S •

Do ask any questions regarding the project or the concepts used in it! 🙂

Collapse
 
osamamalik profile image
Osama Malik •

The rewriter snippet will not run as written. The prompt template interpolates user_query, but invoke passes a dict whose key is question. ChatPromptTemplate raises on the missing input variable before the model is called. Pass user_query, or rename the placeholder to question, and keep that name aligned with the graph state. One rewritten string still goes through a single similarity search. If the guest asks about airport pickup and breakfast, the nearest chunks can all be about pickup, and breakfast never enters the prompt. Splitting into one search per part, then merging the hits, is the usual fix once the rewriter has already found the parts.

Collapse
 
zeroshotanu profile image
Ananya S •

Thank you for catching that! You are completely right—the template placeholder and the .invoke() key were mismatched. I have updated the snippet in the post so they align properly.

Your point on multi-part queries is spot on as well. A single rewritten string still suffers from the top-k chunks clustering around just one part of the question (like airport pickup) while completely crowding out the other (breakfast).

I am definitely aware of that limitation with a single similarity search. Since this series documents my step-by-step progression rather than jumping straight to the final architecture, I wanted to walk through each strategy as I added and tested it.

I actually ran into this exact bottleneck when I evaluated the pipeline against a 20-question Golden Dataset. In one of my upcoming posts where I would be discussing my RAG strategies further, I break down how switching the retrieval strategy from standard cosine similarity to MMR (Maximal Marginal Relevance) helped force chunk diversity for those multi-part queries. Combining that with query decomposition (splitting into sub-queries and merging the hits) is a great next step for handling even more complex prompts.
While I did not implement query decomposition, switching to MMR (Maximal Marginal Relevance) noticeably improved my evaluation metrics on multi-part queries. I will be covering that exact progression and the results in my upcoming posts. Appreciate the thoughtful feedback!