DEV Community: Haider Farooq

RAG That Actually Works: A Practical Checklist

Haider Farooq — Fri, 21 Aug 2026 10:12:38 +0000

Retrieval-augmented generation is the most requested AI feature and the most commonly botched. The failure is almost never the generation step -- it's retrieval quietly returning the wrong context, and the model confidently summarizing garbage. This checklist comes from building research and analysis pipelines where wrong retrieval means wrong business decisions.

Chunking: respect the document's own structure

Fixed 512-token windows sliced mid-sentence are the default and the mistake. Chunk on semantic boundaries -- headings, paragraphs, table rows -- and prepend each chunk with its breadcrumb ("Annual Report 2025 > Risk Factors > Currency"). A chunk that can't be understood alone can't be retrieved reliably.

Hybrid search beats embeddings alone

Dense vectors miss exact identifiers -- SKUs, names, error codes -- that keyword search catches trivially. Run BM25 and vector search in parallel and fuse results (reciprocal rank fusion is fine). This one change fixes a huge share of "the answer was in the corpus but retrieval missed it" tickets.

Rerank before you generate

First-stage retrieval optimizes recall; a cross-encoder reranker optimizes precision on the top 30-50 candidates. Feeding the model 6 highly relevant chunks beats feeding it 20 mediocre ones -- better answers and lower token spend.

Metadata filters are half the battle

Store structured fields (date, source, doc type, customer) alongside every chunk.
Resolve filters before semantic search -- "latest pricing" should hard-filter on recency, not hope embeddings notice.
Let the LLM extract the filter from the question with a typed schema, then apply it in the database.

Answer honesty: refuse to guess

The system prompt must make abstention a first-class outcome: if the retrieved context doesn't contain the answer, say so and show what was found. Require citations to chunk IDs and render them in the UI. Users forgive "not found"; they don't forgive confident fabrication.

Evaluate retrieval separately from generation

Build a golden set of ~50 real questions with known source passages.
Measure retrieval hit-rate (is the right chunk in the top-k?) -- this isolates most failures.
Then grade end-to-end answers for faithfulness and completeness.
Re-run on every change to chunking, embeddings, or prompts.

RAG is a search-quality problem wearing an AI costume. Treat it with the same rigor as the rest of your production agent stack, and it stops being the feature users quietly distrust.

Originally published at haiderfarooq.dev.

7 Lessons from Building Agentic AI in Production

Haider Farooq — Fri, 21 Aug 2026 10:01:55 +0000

For the past year I've been a core engineer on TryCook.ai, an AI operating system that replaced a $3M/year fulfillment team and powers 348+ founders. That means agents doing real, billable work every day -- not demos. Here are the seven lessons that survived contact with production.

1. The agent is 20% of the system

The LLM call is the easy part. The other 80% is queues, retries, idempotency keys, state machines, audit logs, and rollback paths. If you architect the agent as a stateless function inside a boring, observable pipeline, failures become recoverable. If the agent is the pipeline, every hallucination is an outage.

2. Constrain outputs structurally, not rhetorically

Prompts that beg the model to "only respond with valid JSON" fail at scale. Schema-enforced outputs (tool calling, JSON mode, grammar constraints) fail approximately never. Every agent boundary in our stack is a typed contract -- if the model can't fill the schema, that's a retry, not a parse error downstream.

3. Design for the 2% failure case first

An agent that's 98% autonomous still fails dozens of times a day at volume. The difference between a product and a liability is what happens on failure: does the task land in a human review queue with full context, or does it silently vanish? Build the escalation path before you build the happy path.

4. Small models for routing, big models for judgment

Most agent steps are classification, extraction, or formatting -- cheap-model work. Reserve frontier models for the few steps that need actual judgment. This is the core of cost engineering for LLM products, and it routinely cuts spend 5-10x with no quality loss.

5. Memory is a database problem, not a prompt problem

Long-running agents need durable state: what they did, what they learned, what the customer prefers. Stuffing history into the context window doesn't scale and isn't queryable. Model agent memory as normal rows -- events, facts, preferences -- and retrieve selectively. Retrieval discipline is the same skill as building RAG that works.

6. Evals are your regression tests

Every prompt change ships against a fixed eval set of real (anonymized) cases with graded outputs. Without evals, prompt engineering is vibes; with them, it's engineering. Start with 30 cases and a pass/fail rubric -- you can grow sophistication later, but you can't retrofit confidence.

7. Humans are a feature, not a fallback

The systems that win don't remove humans -- they move humans up the stack. In a well-designed agentic system, one operator supervises the throughput that used to take a team, intervening only on flagged exceptions. Frame it that way and adoption stops being a fight.

Agents don't replace engineering discipline. They punish the lack of it, at machine speed.

Originally published at haiderfarooq.dev.

How to Get Cited by AI Search Engines: A Practical GEO Checklist

Haider Farooq — Fri, 21 Aug 2026 09:59:59 +0000

Search is splitting in two. Classic Google still matters, but a growing share of discovery now happens inside ChatGPT, Perplexity, Claude, and Google's AI Overviews. Those engines don't rank ten blue links — they synthesize an answer and cite a handful of sources. Getting cited is the new ranking. The practice of engineering for it is called Generative Engine Optimization (GEO), and this post documents the exact setup running on my site.

1. Let the AI crawlers in

Every AI engine has its own crawler, and many sites block them by accident with a blanket rule. Your robots.txt should explicitly allow the bots you want: GPTBot and OAI-SearchBot (OpenAI), PerplexityBot, ClaudeBot and anthropic-ai (Anthropic), Google-Extended (Gemini training), Applebot-Extended (Apple Intelligence), and CCBot (Common Crawl, which feeds many models).

User-Agent: GPTBot
User-Agent: OAI-SearchBot
Allow: /

User-Agent: PerplexityBot
Allow: /

User-Agent: ClaudeBot
Allow: /

Sitemap: https://haiderfarooq.dev/sitemap.xml

2. Ship an llms.txt

llms.txt is an emerging convention: a plain-markdown file at your site root that gives language models a compressed, structured summary of who you are and what your pages contain. Think of it as a sitemap for meaning rather than URLs. Keep it under a few hundred lines, lead with a one-sentence identity statement, and list your key pages with one-line summaries each. A fuller llms-full.txt can carry the long version.

3. Build an entity graph with JSON-LD

AI engines resolve entities, not keywords. Schema.org JSON-LD is how you declare them unambiguously. The minimum viable graph for a personal site: a Person node with @id, sameAs links to GitHub/LinkedIn/X, knowsAbout, and hasOccupation; a WebSite node; and CreativeWork or Article nodes for project pages. Crucially, use stable @id URIs and reference them across pages so crawlers can stitch the graph together.

{
  "@type": "Person",
  "@id": "https://haiderfarooq.dev/#person",
  "name": "Haider Farooq",
  "jobTitle": "AI Engineer & Data Scientist",
  "sameAs": [
    "https://github.com/haiderfarooq3",
    "https://www.linkedin.com/in/haiderfarooqdev/"
  ]
}

4. Write outcome-first copy

LLMs extract and repeat concrete numbers. "$12.1M in client revenue", "141 districts", "5,500+ patients" — figures like these survive summarization; adjectives don't. Put at least one hard number in the first 150 words of every page. Write bios in third person ("Haider Farooq is an AI engineer…") because that's the frame AI answers use when citing you.

5. Freshness and internal linking still matter

Update pages on a real cadence — stale lastmod dates in your sitemap depress recrawl priority.
Interlink related pages with descriptive anchor text; AI crawlers follow links to build context just like Googlebot does.
Use IndexNow to ping Bing (which feeds ChatGPT search and Copilot) the moment content changes.
Keep canonical URLs consistent — one host, one protocol, no trailing-slash ambiguity.

The checklist

robots.txt allows GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended, CCBot
llms.txt + llms-full.txt at site root
JSON-LD Person/WebSite/Article graph with stable @id URIs
One concrete metric in the first 150 words of every page
Sitemap with honest lastmod dates, submitted to Google Search Console and Bing
IndexNow key installed and pinged on every deploy

None of this replaces good content — it makes good content legible to machines.

Originally published at haiderfarooq.dev.