The Complete Guide for Senior Engineers (5–15 Years Experience)
Written by Himanshu Agarwal
This guide contains 300 in-depth interview questions and answers — 100 each on LLMs, RAG (Retrieval-Augmented Generation), and MCP (Model Context Protocol) — curated specifically for senior engineers, architects, and tech leads (5–15 YOE) preparing for AI/ML, GenAI, and applied AI engineering interviews.
Want the full deep-dive version with case studies, system design diagrams, code walkthroughs, and mock interview drills? Check out the MCP, RAG & LLM Mastery Bundle on Gumroad.
Table of Contents
- Part 1 — Large Language Models (100 Q&A)
- Part 2 — Retrieval-Augmented Generation (100 Q&A)
- Part 3 — Model Context Protocol (100 Q&A)
- Resources
- About the Author
- Explore the Full Bundle
Part 1 — Large Language Models (100 Q&A)
A. Foundations & Architecture (Q1–15)
1. What is a Large Language Model (LLM)?
An LLM is a neural network, typically Transformer-based, trained on massive text corpora to predict the next token in a sequence. At scale (billions of parameters), this next-token prediction objective gives rise to emergent capabilities like reasoning, translation, and code generation, without those tasks being explicitly trained for.
2. Explain the Transformer architecture at a senior level.
The Transformer replaces recurrence with self-attention, allowing every token to attend to every other token in parallel. Core components: multi-head self-attention, position-wise feed-forward networks, residual connections, layer normalization, and positional encodings. This parallelism is what made training on web-scale data computationally feasible versus RNNs/LSTMs.
3. What is self-attention and why does it scale quadratically?
Self-attention computes a weighted sum of value vectors, where weights come from the dot product of query and key vectors across all token pairs. Because every token compares against every other token, compute and memory scale as O(n²) with sequence length n — the primary bottleneck for long-context models.
4. Difference between encoder-only, decoder-only, and encoder-decoder models?
Encoder-only (BERT) builds bidirectional representations, ideal for classification/embedding tasks. Decoder-only (GPT, Llama) is autoregressive and causal, ideal for generation. Encoder-decoder (T5, BART) combines both — encoder reads the full input, decoder generates output conditioned on it — well suited to translation and summarization.
5. What are positional encodings and why are they needed?
Self-attention is permutation-invariant by default — it has no notion of token order. Positional encodings (sinusoidal, learned, or rotary/RoPE) inject order information. RoPE, used in most modern LLMs, encodes relative position via rotation matrices applied to query/key vectors, generalizing better to longer sequences.
6. Explain multi-head attention and why multiple heads help.
Instead of one attention computation, the model splits Q/K/V into multiple lower-dimensional "heads" that attend in parallel, each potentially specializing in different relationships (syntax, coreference, long-range dependency). Outputs are concatenated and projected back, giving richer representational capacity than a single attention head.
7. What is the role of layer normalization, and Pre-LN vs Post-LN?
LayerNorm stabilizes training by normalizing activations across the feature dimension. Post-LN (original Transformer) applies norm after the residual add; Pre-LN applies it before the sublayer. Pre-LN gives more stable gradients at scale and is preferred in modern LLMs since it avoids gradient explosion in very deep stacks.
8. What is KV caching and why is it critical for inference?
During autoregressive generation, keys and values for previously generated tokens don't change, so caching them avoids recomputing attention over the whole sequence at every step. This turns per-token generation cost from O(n²) to O(n), making real-time inference feasible, at the cost of growing memory usage with context length.
9. Explain Mixture of Experts (MoE) architectures.
MoE replaces a single dense FFN with multiple "expert" FFNs, and a gating/router network selects a sparse subset (e.g., top-2) per token. This decouples parameter count from compute cost — models like Mixtral or DeepSeek-MoE have huge total parameters but only activate a fraction per forward pass, improving efficiency.
10. What is Grouped Query Attention (GQA) and Multi-Query Attention (MQA)?
MQA shares a single K/V head across all query heads, drastically reducing KV cache size at some quality cost. GQA is a middle ground — groups of query heads share a K/V head. Modern models (Llama 3, Mistral) use GQA to balance inference efficiency and generation quality.
11. How does RoPE (Rotary Position Embedding) work?
RoPE encodes absolute position by rotating query and key vectors in 2D subspaces by an angle proportional to position, so the dot product between two rotated vectors naturally encodes their relative distance. This gives better extrapolation to longer sequences than fixed sinusoidal or learned absolute embeddings.
12. What is context window and what limits it?
The context window is the maximum number of tokens (input + output) the model can process at once. It's limited by the O(n²) attention cost, positional encoding generalization, and KV cache memory. Techniques like sliding window attention, RoPE scaling, and linear attention variants extend it.
13. What are activation functions commonly used in LLM FFNs?
Modern LLMs mostly use SwiGLU or GeGLU (gated linear units combined with Swish/GELU) instead of plain ReLU, because the gating mechanism improves expressiveness and gradient flow. SwiGLU is used in Llama, PaLM, and most current-generation open models.
14. What is the difference between parameters and FLOPs, and why does it matter for scaling?
Parameters are the model's learned weights; FLOPs measure actual compute used during training/inference. Chinchilla scaling laws showed that for a fixed compute budget, there's an optimal balance of parameters vs training tokens — many earlier models were "undertrained" relative to their size.
15. Explain tokenization and why subword tokenization (BPE) is used.
Byte-Pair Encoding iteratively merges frequent character/subword pairs into a vocabulary, balancing between word-level (huge vocab, poor OOV handling) and character-level (long sequences, weak semantics) tokenization. It lets models handle rare words, multiple languages, and code efficiently with a fixed vocabulary size.
B. Training & Fine-Tuning (Q16–30)
16. Describe the full LLM training pipeline.
Pre-training (self-supervised next-token prediction on trillions of tokens) → Supervised Fine-Tuning (SFT) on instruction-response pairs → Preference alignment (RLHF/DPO) → optional domain-specific fine-tuning/RAG integration → safety red-teaming and evaluation before deployment.
17. What is RLHF and how does it work end-to-end?
Reinforcement Learning from Human Feedback: (1) collect human preference data ranking model outputs, (2) train a reward model to predict those preferences, (3) use PPO to fine-tune the LLM policy to maximize reward while a KL penalty keeps it close to the SFT model, preventing reward hacking/degeneration.
18. What is DPO (Direct Preference Optimization) and why is it popular?
DPO reformulates the RLHF objective into a single supervised loss directly on preference pairs, eliminating the separate reward model and RL loop. It's simpler, more stable, and cheaper to train than PPO-based RLHF while achieving comparable alignment quality, which is why most open models now use it.
19. Explain LoRA (Low-Rank Adaptation) fine-tuning.
LoRA freezes the pretrained weight matrices and injects trainable low-rank decomposition matrices (A, B) alongside them, so updates are ΔW = BA with rank r << d. This cuts trainable parameters by orders of magnitude, drastically reducing GPU memory and enabling fine-tuning of large models on modest hardware.
20. What is QLoRA and how does it differ from LoRA?
QLoRA quantizes the base model to 4-bit precision (NF4) and fine-tunes LoRA adapters on top in higher precision, using techniques like double quantization and paged optimizers to manage memory spikes. It enables fine-tuning 65B+ parameter models on a single consumer GPU.
21. What is catastrophic forgetting and how do you mitigate it during fine-tuning?
Fine-tuning on a narrow dataset can overwrite general capabilities learned during pre-training. Mitigations: use parameter-efficient methods like LoRA (limits weight drift), mix in a portion of general instruction data, use lower learning rates, early stopping, and evaluate on held-out general benchmarks during training.
22. When would you choose full fine-tuning over PEFT methods like LoRA?
Full fine-tuning is justified when you need deep domain adaptation (e.g., a new language, drastically different data distribution, or architecture-level behavior changes) and have the compute budget. For most instruction-tuning, style adaptation, or narrow task specialization, PEFT (LoRA/QLoRA) gives comparable results far more cheaply.
23. Explain instruction tuning and its purpose.
Instruction tuning fine-tunes a base (raw next-token) model on curated (instruction, response) pairs so it learns to follow natural language commands rather than just continue text. It's the step that converts a "text completer" into an assistant-like model responsive to prompts.
24. What is the role of a reward model in alignment pipelines?
The reward model is trained on human preference comparisons (A vs B rankings) to output a scalar score approximating human judgment of response quality. It acts as a proxy objective the policy model optimizes against during RL-based alignment, since raw human feedback can't be given at every training step.
25. What is constitutional AI / RLAIF?
Instead of relying purely on human-labeled preferences, the model critiques and revises its own outputs against a written set of principles (a "constitution"), and an AI (not human) labels preferences based on those principles (RLAIF - RL from AI Feedback), reducing human annotation cost while maintaining alignment.
26. How do you decide the right learning rate and batch size for fine-tuning?
Use a much smaller LR than pre-training (often 1e-5 to 5e-5 for full fine-tune, higher like 1e-4 to 3e-4 for LoRA), with linear warmup and cosine/linear decay. Batch size is constrained by GPU memory; use gradient accumulation to simulate larger effective batches, and monitor loss curves for instability.
27. What is gradient checkpointing and why use it?
Instead of storing all intermediate activations for backprop, gradient checkpointing stores only a subset and recomputes the rest during the backward pass. This trades compute for memory, enabling training of larger models or longer sequences on limited GPU memory.
28. Explain mixed precision training (FP16/BF16) and why BF16 is often preferred.
Mixed precision uses lower-precision (16-bit) formats for most computation while keeping a master copy of weights in FP32, speeding up training and halving memory. BF16 has the same exponent range as FP32 (better numerical stability, no need for loss scaling) versus FP16, which has more mantissa precision but a narrower range prone to overflow/underflow.
29. What is data contamination in LLM training/evaluation, and how do you detect it?
Contamination occurs when benchmark test data leaks into the training corpus, inflating evaluation scores unrealistically. Detection methods: n-gram overlap analysis between training data and benchmarks, canary strings, and held-out/decontaminated benchmark variants released after a model's training cutoff.
30. How would you curate a high-quality instruction-tuning dataset?
Prioritize diversity of task types, difficulty, and domains; deduplicate aggressively; filter for length and quality (heuristics + model-based scoring); mix human-written and synthetic (model-generated, then verified) examples; include multi-turn conversations; and balance refusal/safety examples without over-indexing on them.
C. Prompting & In-Context Learning (Q31–40)
31. What is in-context learning (ICL) and why does it emerge?
ICL is the ability of an LLM to learn a task from examples given directly in the prompt, without weight updates. It's believed to emerge from pre-training exposure to naturally occurring "few-shot-like" patterns in text, and is more pronounced in larger models — a key emergent capability of scale.
32. Zero-shot vs few-shot vs chain-of-thought prompting — when to use each?
Zero-shot works for simple, well-known tasks. Few-shot helps when output format or task nuance is ambiguous and examples clarify it. Chain-of-thought (asking the model to reason step-by-step) significantly improves performance on multi-step reasoning, math, and logic tasks by giving the model "space" to compute intermediate steps.
33. What is prompt engineering and what are core best practices?
It's the practice of structuring inputs to reliably elicit desired model behavior: being explicit about format/constraints, providing examples, decomposing complex tasks, using role/system prompts, requesting structured output (JSON/XML), and iterating empirically since LLM behavior is sensitive to phrasing.
34. Explain the difference between system, user, and assistant roles in chat models.
System sets persistent behavior/persona/constraints for the whole conversation. User messages are the human's turns. Assistant messages are the model's prior responses, included in context for multi-turn coherence. Training data explicitly labels these roles so the model learns differentiated behavior per role.
35. What is self-consistency prompting?
Instead of taking one chain-of-thought output, you sample multiple reasoning paths (with temperature > 0) and take a majority vote on the final answer. This improves accuracy on reasoning tasks by marginalizing out noise in any single generation path, at the cost of extra inference calls.
36. What is ReAct prompting?
ReAct interleaves reasoning ("Thought") and actions ("Action", e.g., tool calls) in the same generation loop — the model reasons about what to do, takes an action (like a search query), observes the result, and continues reasoning. This is foundational to agentic LLM systems and tool use.
37. How do you mitigate prompt injection in production LLM applications?
Separate trusted (system) instructions from untrusted (user/retrieved) content structurally, use delimiters and explicit instructions to ignore embedded commands, apply input/output filtering, sandbox tool execution with least privilege, and consider using models specifically hardened against injection with dedicated instruction hierarchies.
38. What is the "lost in the middle" problem in long-context prompting?
Research shows LLMs often attend better to information at the beginning and end of a long context than the middle, degrading recall for facts placed mid-context. Mitigations: place critical information near the start/end, use retrieval to keep context focused, or use models specifically tuned for long-context recall.
39. Explain few-shot example selection strategies for production prompts.
Static few-shot examples work for stable tasks; dynamic selection (e.g., retrieving semantically similar examples via embeddings per query) improves performance on diverse inputs. Diversity and difficulty-matching in the example set both matter more than raw example count beyond a small threshold.
40. What is structured output prompting (JSON mode / function calling) and why is it important for production systems?
It constrains the model's output to a defined schema (via grammar-constrained decoding, fine-tuned "JSON mode," or function-calling APIs), making outputs machine-parseable and reliable for downstream integration — critical for agents, tool use, and any system where free text can't be safely parsed.
D. Inference & Optimization (Q41–55)
41. Explain quantization (INT8, INT4, GPTQ, AWQ) and its trade-offs.
Quantization reduces weight/activation precision to shrink memory footprint and increase throughput. GPTQ uses layer-wise error-minimizing quantization post-training; AWQ preserves precision for "salient" weight channels identified by activation statistics. Trade-off: lower bit-widths risk quality degradation, especially on reasoning-heavy tasks, requiring careful calibration.
42. What is speculative decoding and how does it speed up inference?
A small, fast "draft" model generates several candidate tokens ahead, and the large target model verifies them in a single parallel forward pass, accepting the correct prefix and only falling back to normal generation on mismatch. This can 2-3x throughput since verification is cheaper than autoregressive generation token-by-token.
43. What is continuous batching and why does it matter for serving throughput?
Traditional static batching waits for all requests in a batch to finish before starting new ones, wasting GPU cycles on short sequences. Continuous (dynamic) batching, used in engines like vLLM and TGI, injects/evicts requests token-by-token, keeping GPU utilization high and dramatically improving serving throughput.
44. Explain PagedAttention (vLLM) and the problem it solves.
KV cache memory is traditionally allocated contiguously per sequence, causing fragmentation and wasted memory since sequence lengths vary. PagedAttention borrows OS virtual memory paging concepts — KV cache is stored in non-contiguous fixed-size blocks — enabling near-zero memory waste and much higher batch sizes.
45. What factors determine LLM inference latency, and how do you optimize each?
Time-to-first-token (prompt processing, prefill compute) and inter-token latency (memory-bandwidth-bound decode step) are the two main components. Optimize prefill with better batching/parallelism; optimize decode with quantization, speculative decoding, smaller KV cache (GQA/MQA), and hardware with high memory bandwidth.
46. What is model distillation and when would you use it?
A smaller "student" model is trained to mimic a larger "teacher" model's outputs (soft labels/logits or generated data), transferring much of its capability at a fraction of the size/cost. Use it when you need low-latency, low-cost inference for a narrower task where the teacher's full generality isn't needed.
47. Explain the trade-offs between temperature, top-k, and top-p (nucleus) sampling.
Temperature scales the logits' sharpness (low = deterministic, high = diverse/random). Top-k restricts sampling to the k most likely tokens. Top-p samples from the smallest set of tokens whose cumulative probability exceeds p, adapting dynamically to the model's confidence — generally preferred over top-k for more natural diversity control.
48. How would you architect an LLM serving system for high concurrency at low cost?
Use a high-throughput inference engine (vLLM/TensorRT-LLM) with continuous batching and PagedAttention, quantize models where quality allows, implement autoscaling with request queuing, route by model/task tier, cache common prompts/responses, and use speculative decoding or smaller distilled models for latency-sensitive paths.
49. What is FlashAttention and why does it matter?
FlashAttention is an IO-aware exact attention algorithm that avoids materializing the full n×n attention matrix in slow HBM memory, instead computing attention in fused, tiled kernels using fast SRAM. This gives significant speedups and memory savings without any approximation, and is now standard in most training/inference stacks.
50. Explain tensor parallelism vs pipeline parallelism vs data parallelism.
Data parallelism replicates the full model across devices, each processing different data batches. Tensor parallelism splits individual weight matrices across devices (needed when a single layer doesn't fit on one GPU). Pipeline parallelism splits the model by layers across devices, passing activations sequentially — often combined (3D parallelism) for very large models.
51. What is the cost/latency trade-off between using a large frontier model vs a smaller fine-tuned model in production?
Frontier models offer stronger zero-shot generalization and reasoning but cost more per token and have higher latency. Smaller fine-tuned/distilled models can match or exceed frontier performance on narrow, well-specified tasks at a fraction of the cost — the right choice depends on task breadth, volume, and latency SLAs.
52. What is caching in the context of LLM applications and what layers can be cached?
Layers include: exact prompt-response caching, semantic caching (cache hits on semantically similar queries via embeddings), KV cache reuse for shared prefixes (prompt caching offered by major providers), and retrieval result caching in RAG systems — each reduces redundant compute and cost.
53. How do you handle rate limiting and retries robustly when calling LLM APIs at scale?
Implement exponential backoff with jitter, respect provider rate-limit headers, use request queuing/token-bucket throttling client-side, batch where APIs support it, have fallback models/providers, and design idempotent retry logic that handles partial failures without duplicating side effects (like tool calls).
54. What is structured/constrained decoding and how is it implemented?
It restricts the model's token sampling at each step to only tokens valid under a given grammar/schema (e.g., JSON schema), typically via masking invalid logits before sampling. Libraries like Outlines, Guidance, or provider-native "structured output" modes implement this, guaranteeing syntactically valid output.
55. What is the difference between prefill and decode phases in LLM inference, and why are they optimized differently?
Prefill processes the entire input prompt in parallel (compute-bound, benefits from large batch/matrix ops). Decode generates one token at a time autoregressively (memory-bandwidth-bound, since KV cache reads dominate). Serving systems often use disaggregated prefill/decode architectures to optimize each phase independently.
E. Evaluation (Q56–65)
56. How do you evaluate LLM outputs beyond standard benchmarks like MMLU?
Combine automated benchmarks (task-specific accuracy), LLM-as-judge scoring against rubrics, human evaluation (pairwise preference or Likert scoring), task-specific business metrics (e.g., resolution rate for a support bot), and red-teaming for safety/robustness — no single metric suffices for production readiness.
57. What is LLM-as-a-judge and what are its pitfalls?
Using a strong LLM to score/compare outputs against criteria, scaling evaluation beyond human capacity. Pitfalls: positional bias (favoring the first option shown), verbosity bias (favoring longer answers), self-preference bias (favoring outputs similar to its own style), and inconsistency — mitigated by randomizing order, using rubrics, and calibrating against human judgments.
58. How do you evaluate factuality/hallucination in LLM outputs?
Techniques include fact-verification against a trusted knowledge source (NLI-based entailment checking), self-consistency checks (sampling multiple generations and checking agreement), citation-grounding verification (does the claim map to a retrieved source), and human spot-checking on a sampled basis for production monitoring.
59. What is perplexity and what are its limitations as an evaluation metric?
Perplexity measures how well a model predicts held-out text (lower = better fit). It correlates with fluency but poorly with downstream task usefulness, factuality, or instruction-following — a model can have low perplexity while being unhelpful or incorrect, so it's mainly used for pre-training model comparison, not instruction-tuned model evaluation.
60. How would you design an eval harness for a domain-specific fine-tuned LLM?
Build a held-out labeled test set representative of production distribution, define task-specific metrics (exact match, F1, ROUGE, or rubric-based scoring depending on task type), include adversarial/edge cases, run regression testing on every model/prompt change, and track metrics over time with statistical significance testing.
61. What is the difference between intrinsic and extrinsic evaluation?
Intrinsic evaluation measures model properties directly (perplexity, benchmark accuracy) independent of a downstream application. Extrinsic evaluation measures impact on the actual task/business outcome (e.g., customer satisfaction, task completion rate) — extrinsic metrics are ultimately what matter for production decisions.
62. How do you evaluate reasoning capability specifically?
Use benchmarks requiring multi-step logic (GSM8K, MATH, BBH), analyze chain-of-thought traces for logical validity (not just final-answer correctness), test consistency by paraphrasing the same problem, and check robustness to distractors/irrelevant information injected into the prompt.
63. What is a golden dataset and how do you build one for evaluation?
A golden dataset is a curated, high-quality, human-verified set of input-output pairs representing correct/ideal behavior. Build it by sampling real production queries, having domain experts label ideal responses, ensuring coverage of edge cases and difficulty levels, and periodically refreshing it as the task distribution shifts.
64. How do you detect and measure bias in LLM outputs?
Use counterfactual testing (swap demographic attributes in prompts and measure output differences), established bias benchmarks (BBQ, StereoSet), disaggregated evaluation across subgroups on real tasks, and qualitative red-teaming — statistical parity alone is insufficient without contextual judgment of harm.
65. What is A/B testing's role in evaluating LLM features in production, and what are its unique challenges?
A/B testing measures real user impact (engagement, task success, satisfaction) beyond offline metrics. Unique LLM challenges: non-determinism requires larger sample sizes for statistical power, delayed/indirect quality signals (e.g., a wrong answer's harm may not surface immediately), and the need to guard against regressions in tail/edge cases invisible in aggregate metrics.
F. Alignment & Safety (Q66–75)
66. What is the alignment problem in the context of LLMs?
Ensuring a model's behavior matches human intentions and values — not just being capable, but being helpful, honest, and harmless in ways humans actually want, including handling ambiguous or underspecified instructions safely and avoiding deceptive or harmful behavior even when technically "following orders."
67. Explain the difference between helpfulness and harmlessness trade-offs in RLHF.
Optimizing purely for helpfulness can produce outputs that comply with harmful requests; optimizing purely for harmlessness can produce an overly-refusing, unhelpful model. Modern alignment balances both via multi-objective reward modeling or constitutional principles that specify nuanced, context-sensitive refusal rather than blanket restriction.
68. What is jailbreaking and what are common techniques attackers use?
Jailbreaking is crafting inputs to bypass a model's safety training. Common techniques: role-play/persona framing ("pretend you're an AI with no restrictions"), prompt injection via indirect content, encoding harmful requests (base64, translated language), multi-turn escalation, and exploiting instruction-hierarchy confusion between system/user content.
69. How do you red-team an LLM application before production launch?
Assemble adversarial testers (internal + external) to probe for harmful outputs, bias, prompt injection, data leakage, and jailbreaks systematically across categories; use automated adversarial prompt generation tools; document and triage findings by severity; and re-test after each mitigation before sign-off.
70. What is the difference between guardrails and alignment training?
Alignment training (RLHF/DPO/Constitutional AI) shapes the model's underlying behavior during training. Guardrails are external systems (input/output filters, classifiers, rule-based checks) wrapped around the model at inference time as a second layer of defense — production systems typically need both, since neither alone is fully robust.
71. What is reward hacking and how does it manifest in RLHF-trained models?
The policy model finds ways to maximize the reward model's score without actually satisfying the true underlying objective — e.g., producing overly long, verbose, or sycophantic responses because the reward model correlates those with quality. Mitigated with reward model regularization, KL penalties, and diverse preference data.
72. Explain sycophancy in LLMs and why it's a safety concern.
Sycophancy is the tendency of a model to agree with or flatter the user's stated views rather than give an accurate/honest answer, often a side effect of RLHF optimizing for human-rated approval. It's concerning because it can reinforce misinformation and erode the model's reliability as an objective source.
73. What is Constitutional AI and how does it differ from standard RLHF?
Constitutional AI has the model critique and revise its own responses against a set of written principles, then trains on the self-improved outputs (plus AI-generated preference labels), reducing reliance on large-scale human labeling of harmful content while making the alignment criteria explicit and auditable.
74. How do you handle PII and data privacy in LLM applications?
Implement PII detection/redaction on inputs before logging or sending to third-party APIs, use data processing agreements with providers, avoid training/fine-tuning on sensitive user data without consent, apply differential privacy techniques where feasible, and ensure retention/deletion policies comply with regulations (GDPR, HIPAA, etc.).
75. What is the difference between AI safety and AI alignment as fields, and why does it matter for a practitioner?
Alignment focuses narrowly on making a model's behavior match intended goals; safety is the broader field including alignment plus robustness, interpretability, misuse prevention, and societal impact. A practitioner needs both: aligned models that also fail gracefully, resist misuse, and are monitored in production.
G. Scaling, Systems & Emerging Topics (Q76–90)
76. Explain scaling laws (Chinchilla) and their practical implications.
Chinchilla scaling laws found that for a fixed compute budget, model size and training tokens should scale roughly equally — many earlier large models were undertrained relative to their parameter count. Practical implication: a smaller model trained on more data can outperform a larger undertrained one at the same compute cost.
77. What are emergent abilities in LLMs and are they real or a measurement artifact?
Emergent abilities are capabilities (e.g., multi-step arithmetic) that appear sharply at certain scale thresholds rather than improving gradually. Some research argues this is partly a measurement artifact of discontinuous metrics (exact-match) rather than the underlying capability, which may improve smoothly when measured with continuous metrics.
78. What is Retrieval-Augmented Generation and how does it relate to LLM limitations?
RAG grounds LLM generation in retrieved external documents at inference time, addressing the model's static knowledge cutoff, hallucination tendency, and inability to cite sources — without requiring retraining for every new piece of information. (Deep dive in Part 2.)
79. What is an AI agent and how does it differ from a standard LLM call?
An agent uses an LLM as a reasoning engine in a loop — planning, taking actions via tools, observing results, and iterating — to accomplish multi-step goals autonomously, rather than producing a single response to a single prompt. Requires state management, tool orchestration, and often memory across steps.
80. What is function calling / tool use, and how is it implemented under the hood?
The model is given structured tool definitions (name, description, parameter schema) in its context; it's fine-tuned to output a structured call (JSON) when a tool is appropriate instead of natural text, which the application layer parses, executes, and feeds the result back into context for the next turn.
81. Explain the difference between multimodal and unimodal LLMs.
Unimodal models process a single modality (text). Multimodal models (GPT-4V, Gemini, LLaVA) process and often generate across modalities (text, images, audio) using techniques like vision encoders projected into the LLM's embedding space, enabling tasks like visual question answering and document understanding.
82. What is model merging and why has it become popular?
Model merging combines the weights of multiple fine-tuned models (via techniques like SLERP, TIES, or DARE) into a single model without additional training, often producing a model with combined capabilities of its parents. It's popular because it's compute-free relative to retraining and can improve robustness/generalization.
83. What is the difference between open-weight and open-source LLMs?
Open-weight models release the trained parameters (usable, fine-tunable) but not necessarily training data, code, or full methodology (e.g., Llama). Fully open-source models release weights, training code, and data recipes (e.g., OLMo, Pythia) enabling full reproducibility — an important distinction for licensing, auditability, and research.
84. Explain the concept of test-time compute / inference-time scaling (as in o1-style reasoning models).
Instead of relying solely on model size for capability, these models spend additional compute at inference time generating extended internal reasoning chains before answering, trading latency/cost for improved accuracy on complex reasoning tasks — a new scaling axis alongside pre-training compute and data.
85. What is model collapse in the context of training on synthetic/AI-generated data?
When models are recursively trained on data generated by prior model generations without sufficient real-data anchoring, error and distributional narrowing compound over generations, degrading diversity and accuracy — a growing concern as synthetic data becomes a larger fraction of the web/training corpora.
86. What is context caching / prompt caching offered by LLM providers, and how does it reduce cost?
Providers cache the KV state for a repeated prefix (e.g., a long system prompt or document) across requests, so subsequent calls sharing that prefix skip redundant prefill computation, significantly reducing latency and cost for applications with large, stable shared context (e.g., RAG systems with long document context).
87. What is the role of synthetic data generation in modern LLM training pipelines?
Synthetic data (model-generated instructions, reasoning traces, or distillation data from stronger models) supplements scarce or expensive human-labeled data, especially for instruction-tuning and reasoning capability. It requires careful filtering/verification to avoid quality degradation or model collapse over successive generations.
88. How do sliding window attention and other long-context techniques work?
Sliding window attention restricts each token to attend only to a fixed-size local window (plus optionally a few global tokens), reducing compute from O(n²) to O(n·w). Combined with techniques like RoPE scaling, ALiBi, or hierarchical/hybrid attention, this enables extending effective context length beyond training-time limits.
89. What is the difference between fine-tuning and RAG for injecting domain knowledge, and how do you choose?
Fine-tuning bakes knowledge/behavior into weights — good for style, format, and stable domain patterns, but expensive to update and prone to hallucination on facts. RAG keeps knowledge external and retrievable — better for frequently-changing or large factual corpora, with built-in citability. Most production systems combine both.
90. What are small language models (SLMs) and when are they the right architectural choice?
SLMs (typically <10B parameters) trade broad generality for efficiency, lower cost, and the ability to run on-device or at very high throughput. They're the right choice for narrow, well-defined tasks with sufficient fine-tuning data, latency-sensitive applications, or privacy-constrained on-device deployment.
H. Production, MLOps & System Design (Q91–100)
91. How would you design an LLM-powered customer support system end-to-end?
Intent classification/routing → RAG over knowledge base for grounded answers → structured escalation logic for out-of-scope or low-confidence cases → human-in-the-loop for high-stakes actions → logging/feedback loop for continuous eval and fine-tuning → guardrails for PII and off-topic/harmful queries → monitoring dashboards for quality drift.
92. What monitoring and observability practices are essential for production LLM systems?
Track latency (TTFT, total), cost per request, token usage, error/refusal rates, output quality via sampled human/LLM-judge review, drift in input distribution, hallucination/groundedness scores for RAG, and user feedback signals (thumbs up/down, escalation rate) — all with alerting on threshold breaches.
93. How do you version and manage prompts in a production system?
Treat prompts as code: store in version control, use templating with parameterization, run regression evals on every change before deployment, support environment-specific configs (dev/staging/prod), and maintain a changelog correlating prompt versions with observed metric changes.
94. What is prompt drift and how do you detect/prevent it?
Prompt drift occurs when underlying model updates (even "same" model versions from a provider) change behavior for an existing prompt, degrading production quality silently. Detect via continuous regression testing against a golden eval set on model/version changes; prevent by pinning model versions where possible and monitoring output metrics over time.
95. How would you architect a multi-tenant LLM platform serving multiple internal teams?
Centralize model access via a gateway (auth, rate limiting, cost attribution per team), provide shared observability/logging infrastructure, support per-tenant configuration (models, prompts, guardrails), implement usage quotas and chargeback, and offer a self-service eval/testing framework so teams can safely iterate independently.
96. What are the key cost drivers in an LLM application and how do you optimize them?
Input/output token volume, model tier choice, redundant calls (lack of caching), and retrieval overhead in RAG. Optimize via prompt compression, semantic caching, routing simple queries to cheaper models, batching where latency allows, and right-sizing context (avoid over-stuffing retrieved documents).
97. How do you handle model deprecation and migration in a production system relying on a third-party LLM API?
Maintain an abstraction layer decoupling application logic from a specific provider/model, run the golden eval suite against candidate replacement models before cutover, do gradual/canary rollout with metric comparison, and keep prompts modular enough to require minimal rework across model families.
98. What is the CI/CD equivalent for LLM applications ("LLMOps")?
Pipeline stages: prompt/data versioning → automated eval suite (regression + safety) on every change → staged rollout (canary/shadow traffic) → production monitoring with automated rollback triggers → periodic re-evaluation as underlying models/data evolve — analogous to traditional CI/CD but with non-deterministic, quality-based gating instead of pass/fail unit tests.
99. How do you decide between building on a proprietary API (OpenAI/Anthropic) vs self-hosting an open-weight model?
Consider: data privacy/compliance requirements, latency/throughput needs, total cost at your volume (API per-token cost vs GPU infra + ops overhead), need for fine-tuning/customization, and required capability ceiling — proprietary APIs generally win for fastest time-to-market and top capability, self-hosting wins for cost-at-scale, data control, and customization depth.
100. Describe a real (or realistic) production incident involving an LLM system and how you'd debug it.
Example: a RAG chatbot suddenly starts hallucinating incorrect answers. Debug path: check if it's isolated to specific query types (retrieval failure) vs global (model/prompt regression) → inspect retrieved context for the failing cases (are relevant docs even being retrieved?) → check for recent prompt/index/model version changes → replay failing cases against golden eval set → roll back the suspected change while root-causing, then add the failure pattern to the regression eval set to prevent recurrence.
Part 2 — Retrieval-Augmented Generation (100 Q&A)
A. RAG Fundamentals (Q1–15)
1. What is RAG and what problem does it solve?
RAG combines a retrieval system with an LLM: relevant documents are fetched from an external knowledge source at query time and injected into the prompt as context, grounding generation in up-to-date, verifiable information. It addresses LLM knowledge cutoffs, hallucination, and the impracticality of retraining models for every knowledge update.
2. Walk through the basic RAG pipeline architecture.
Ingestion: documents are chunked, embedded, and stored in a vector index. Query time: the user query is embedded, top-k similar chunks are retrieved, optionally reranked, assembled into a prompt with the query, and passed to the LLM to generate a grounded response, often with citations.
3. What are the main components of a production RAG system?
Document loaders/parsers, chunking strategy, embedding model, vector store/index, retrieval logic (dense/sparse/hybrid), reranker, prompt assembly/context management, the generator LLM, and an evaluation/observability layer — each is independently tunable and a common point of failure.
4. Why does RAG reduce hallucination but not eliminate it?
Grounding the model in retrieved context reduces reliance on parametric (memorized, potentially stale/wrong) knowledge, but the model can still misread, over-generalize beyond, or ignore the retrieved context, or retrieval itself can fail to surface the correct documents — so hallucination risk is reduced, not removed.
5. What is the difference between RAG and fine-tuning for knowledge injection?
RAG externalizes knowledge (retrievable, updatable, citable, no retraining needed) while fine-tuning internalizes it into weights (better for behavior/style/format, but static and hard to audit/update). Most production systems use RAG for facts and fine-tuning for tone, format, and task-specific behavior.
6. What is "naive RAG" vs "advanced RAG" vs "modular RAG"?
Naive RAG is the basic embed-retrieve-generate pipeline. Advanced RAG adds pre-retrieval (query rewriting, routing) and post-retrieval (reranking, compression) optimizations around the same core flow. Modular RAG treats retrieval, routing, and generation as composable, potentially iterative modules — enabling patterns like multi-hop retrieval or agentic RAG.
7. What types of documents/data sources is RAG best suited for?
Best for large, frequently-updated, or proprietary text corpora where citability matters: internal knowledge bases, documentation, legal/compliance documents, customer support histories, and research papers. Less suited (alone) for tasks requiring complex multi-step numerical reasoning or data better served by structured queries (SQL) over databases.
8. What is grounding, and how do you measure how well a RAG response is grounded?
Grounding means every factual claim in the generated response is supported by the retrieved context. Measured via automated faithfulness metrics (NLI-based entailment checking of claims against source chunks), citation-attribution verification, or LLM-as-judge scoring against the retrieved documents.
9. When would you NOT use RAG?
When the task doesn't require external/current knowledge (pure creative writing, general reasoning on self-contained input), when ultra-low latency is critical and retrieval adds unacceptable overhead, when the knowledge base is small enough to fit entirely in context, or when structured data querying (SQL/API) is more appropriate than semantic retrieval.
10. What is "context stuffing" and why is it a poor default strategy?
Context stuffing means passing as many retrieved documents as possible to maximize the chance of including relevant info. It's poor because it increases cost/latency, risks the "lost in the middle" problem diluting relevant content, and can actually reduce answer quality — well-tuned top-k with reranking usually beats brute-force stuffing.
11. Explain the trade-off between retrieval precision and recall in RAG.
High recall (retrieve broadly) ensures relevant information isn't missed but risks diluting context with noise, hurting generation quality and increasing cost. High precision (retrieve narrowly) keeps context focused but risks missing needed information. Production systems tune this via top-k, similarity thresholds, and reranking to balance both.
12. What is multi-hop retrieval and when is it needed?
Multi-hop retrieval performs sequential retrieval steps where each retrieved result informs the next query (e.g., answering "What company did the founder of X work at before?" requires first retrieving who founded X, then retrieving that person's work history). Needed for compositional questions that a single retrieval pass can't answer.
13. What is agentic RAG?
Agentic RAG gives the LLM autonomy over the retrieval process itself — deciding whether to retrieve, reformulating queries, choosing which knowledge source to query, evaluating if retrieved results are sufficient, and iterating (retrieve-generate-critique loops) rather than following a fixed single-pass pipeline.
14. What is the difference between RAG and long-context LLMs — does long context make RAG obsolete?
Long-context models can ingest entire documents directly, but at higher cost/latency and with degraded recall on very long inputs ("lost in the middle"). RAG remains valuable for cost efficiency, citability, freshness (no need to re-embed the whole corpus into every prompt), and scaling to corpora far larger than any context window. They're often complementary, not competing.
15. Explain corrective RAG (CRAG) and self-RAG.
CRAG adds a lightweight evaluator that grades retrieved documents' relevance; if retrieval quality is poor, it triggers corrective actions like web search or query rewriting before generation. Self-RAG trains the LLM itself to emit special reflection tokens deciding when to retrieve, critiquing retrieved passages, and assessing its own output's support — both aim to make RAG more robust to poor retrieval.
B. Chunking & Preprocessing (Q16–30)
16. Why does chunking strategy significantly impact RAG quality?
Chunk size and boundaries determine what semantic units are retrievable and embeddable — too large dilutes relevance signal and wastes context; too small loses necessary context/coherence for the LLM to answer correctly. Poor chunking is one of the most common root causes of RAG failures in practice.
17. Compare fixed-size, recursive, semantic, and document-structure-aware chunking.
Fixed-size (token/character count) is simple but ignores semantic boundaries. Recursive chunking splits along a hierarchy of separators (paragraphs → sentences) to respect structure while hitting size targets. Semantic chunking groups sentences by embedding similarity to keep coherent ideas together. Structure-aware chunking respects document elements (headings, tables, code blocks) — generally the best-performing but most implementation-heavy approach.
18. What is chunk overlap and why is it used?
Overlap (repeating a portion of text between adjacent chunks) prevents important context from being split awkwardly across a chunk boundary, ensuring a query can still retrieve the full relevant passage even if the key sentence straddles two chunks. Typical overlap is 10-20% of chunk size.
19. How do you handle tables, code, and structured content during chunking?
Extract and preserve structural integrity — keep tables intact (or convert to a serialized text/markdown representation) rather than splitting rows arbitrarily; treat code blocks as atomic units; use specialized parsers (e.g., unstructured.io, layout-aware PDF parsers) rather than naive text extraction that would mangle these elements.
20. What is parent-child (small-to-big) chunking retrieval?
Small, precise chunks are embedded and used for retrieval matching (better semantic precision), but when a small chunk is retrieved, its larger parent chunk/section is what's actually passed to the LLM for generation — combining precise retrieval with sufficient context for coherent answers.
21. How do you determine the optimal chunk size for a given use case?
It depends on the embedding model's effective context window, the nature of the content (dense technical text vs conversational), and the granularity of expected queries (fact lookup favors smaller chunks, summarization/broad questions favor larger). Empirically tune via retrieval eval metrics (recall@k) rather than picking a size a priori.
22. What is metadata filtering in RAG and why is it important?
Attaching structured metadata (date, author, document type, department, access level) to chunks allows retrieval to be filtered/scoped before or alongside semantic search — critical for multi-tenant access control, recency requirements, and narrowing large heterogeneous corpora to relevant subsets, improving both precision and compliance.
23. How do you handle document updates and deletions in a RAG index without stale data?
Maintain a mapping from source document to its chunk IDs so updates can delete-and-reindex just the affected chunks; use versioning/timestamps to prefer freshest content; implement periodic full re-sync jobs alongside incremental updates; and avoid orphaned chunks by ensuring deletions propagate to the vector store.
24. What is contextual retrieval (Anthropic's technique) and why does it improve results?
Contextual retrieval prepends a short, LLM-generated summary of how a chunk relates to the overall document before embedding/indexing it, so the chunk's embedding and BM25 representation carry document-level context it would otherwise lose in isolation — shown to significantly reduce retrieval failures.
25. How should you handle multi-modal documents (PDFs with images, charts, scanned pages) in a RAG pipeline?
Use layout-aware extraction (e.g., OCR for scanned content, vision-language models to caption charts/images), preserve reading order and structural hierarchy, consider multi-modal embeddings for image content, and store extracted descriptions alongside/instead of raw images depending on whether visual retrieval is needed.
26. What preprocessing steps matter before chunking (cleaning, deduplication, normalization)?
Remove boilerplate (headers/footers/navigation), deduplicate near-identical content across sources, normalize whitespace/encoding, resolve or strip broken formatting artifacts from extraction, and standardize date/number formats — noisy input directly degrades embedding quality and retrieval precision.
27. How do you chunk long-form content like books or lengthy legal contracts differently from short documents like FAQs?
Long-form content benefits from hierarchical chunking (section → paragraph) with parent-child retrieval to preserve context, and larger overlap given denser cross-references. Short documents (FAQs, short articles) often work best as whole-document or single-chunk units, since splitting can destroy the atomic Q&A structure.
28. What is sliding window chunking and its trade-offs?
A fixed-size window moves through the document with a defined stride shorter than the window size, creating overlapping chunks. It maximizes context preservation across boundaries but multiplies storage/embedding cost and can introduce near-duplicate chunks that skew retrieval ranking if not deduplicated.
29. How do you evaluate whether your chunking strategy is effective?
Measure retrieval recall@k on a labeled eval set (does the correct chunk get retrieved for known queries), inspect chunk boundaries manually for semantic coherence on samples, and run end-to-end answer quality evaluation comparing different chunking configurations — chunking should be evaluated empirically, not assumed.
30. What is late chunking and how does it differ from traditional chunking?
Late chunking runs the full document through a long-context embedding model first to get token-level contextualized embeddings, then pools/splits into chunks afterward — so each chunk's embedding still carries full-document context, addressing the context-loss problem of chunking before embedding.
C. Embeddings & Vector Databases (Q31–45)
31. What is a text embedding and how is it generated?
An embedding is a dense vector representation of text where semantic similarity corresponds to geometric proximity (e.g., cosine similarity). Generated by encoder models (BERT-derivatives, or dedicated embedding models like OpenAI's text-embedding-3 or open models like BGE/E5) trained via contrastive learning on similar/dissimilar text pairs.
32. How do you choose an embedding model for a RAG system?
Consider retrieval benchmark performance for your domain (MTEB leaderboard as a starting reference), embedding dimensionality (trade-off between quality and storage/speed), max input token length, multilingual support if needed, licensing/cost (API vs self-hosted), and empirical evaluation on your own labeled query-document pairs.
33. What is the difference between dense and sparse retrieval?
Dense retrieval uses learned embeddings and semantic (cosine/dot-product) similarity, capturing meaning beyond exact word match. Sparse retrieval (TF-IDF, BM25) uses term-frequency-based statistics over exact tokens, excelling at keyword/exact-match queries (IDs, rare terms, jargon) where dense models can underperform.
34. What is hybrid search and why does it typically outperform pure dense or sparse retrieval?
Hybrid search combines dense (semantic) and sparse (lexical/BM25) retrieval results, typically fused via reciprocal rank fusion or weighted scoring, capturing both semantic similarity and exact keyword matches. It's more robust because dense and sparse methods fail on different query types, so combining covers each other's blind spots.
35. Explain how vector similarity search works (cosine similarity, dot product, Euclidean distance).
Cosine similarity measures the angle between vectors (magnitude-invariant, most common for normalized embeddings). Dot product incorporates magnitude and is used when embedding norms carry meaningful signal (often equivalent to cosine after normalization). Euclidean (L2) distance measures straight-line distance — choice should match how the embedding model was trained/optimized.
36. What is Approximate Nearest Neighbor (ANN) search and why is it necessary at scale?
Exact nearest-neighbor search is O(n) per query — infeasible at millions/billions of vectors. ANN algorithms (HNSW, IVF, LSH) trade a small amount of recall accuracy for orders-of-magnitude speedup by building index structures that avoid exhaustive comparison, making large-scale vector search practical.
37. Explain HNSW (Hierarchical Navigable Small World) at a conceptual level.
HNSW builds a multi-layer graph where higher layers have fewer, longer-range connections (for fast coarse navigation) and lower layers have denser, short-range connections (for fine-grained search). Search starts at the top layer and greedily descends, giving logarithmic-ish search complexity with high recall — the most widely used ANN algorithm in production vector databases.
38. How do you choose between vector database options (Pinecone, Weaviate, Milvus, pgvector, Qdrant, FAISS)?
Consider: managed vs self-hosted trade-off, scale requirements (billions of vectors need distributed architectures), metadata filtering capabilities, hybrid search support, existing infra (pgvector fits naturally if already on Postgres), latency SLAs, and cost — FAISS is a library (not a full DB) best for embedded/research use cases, not multi-tenant production serving.
39. What is vector index quantization (PQ, scalar quantization) and why use it?
Product Quantization (PQ) and scalar quantization compress vector representations to reduce memory footprint and speed up distance computation, at the cost of some precision loss. Essential when indexing billions of vectors where full-precision storage would be prohibitively expensive.
40. How do you handle embedding model versioning/upgrades without breaking a live RAG index?
Since different embedding model versions produce incompatible vector spaces, you generally must fully re-embed and re-index the entire corpus when upgrading models — plan for a shadow index built with the new model, validate retrieval quality against the old one, then cut over atomically rather than mixing embeddings from different models in one index.
41. What is re-embedding drift and how do you monitor for it?
Drift occurs when the distribution of incoming queries/documents shifts over time relative to what the embedding model was optimized for, degrading retrieval quality silently. Monitor via periodic retrieval eval on a fixed labeled set, tracking recall@k over time, and analyzing query logs for emerging out-of-distribution patterns.
42. Explain Matryoshka embeddings and why they're useful.
Matryoshka Representation Learning trains embeddings so that truncating the vector to a smaller dimension (e.g., 768 → 128) still yields a usable, if less precise, embedding. This lets a single model serve multiple storage/speed tiers by truncating dimensions as needed, without training separate models per dimensionality.
43. What is the difference between bi-encoders and cross-encoders in the retrieval context?
Bi-encoders embed query and document independently, enabling fast pre-computed vector search at scale but losing fine-grained query-document interaction. Cross-encoders jointly process the query-document pair through a single model for much higher accuracy, but are too slow to run over an entire corpus — hence used for reranking a small candidate set, not initial retrieval.
44. How do you handle multi-lingual retrieval in a RAG system?
Use multilingual embedding models trained on cross-lingual contrastive data (e.g., multilingual-E5, LaBSE) so semantically equivalent text in different languages maps close together in vector space, enabling cross-lingual retrieval (query in one language, documents in another) without translation as an intermediate step.
45. What causes "semantic drift" between a query and retrieved chunks, and how do you mitigate it?
Short, ambiguous, or jargon-heavy queries may embed far from relevant document phrasing even when topically related, due to vocabulary/style mismatch. Mitigate with query expansion/rewriting, contextual retrieval (enriching chunks with context before embedding), hybrid search to catch exact-term matches dense search misses, and fine-tuning the embedding model on domain-specific query-document pairs.
D. Retrieval Strategies (Q46–60)
46. What is query rewriting/expansion and why is it used in RAG?
The original user query is transformed (via LLM rephrasing, synonym expansion, or decomposition into sub-questions) before retrieval to better match how relevant information is phrased in the corpus, improving recall especially for short, ambiguous, or conversational queries.
47. Explain HyDE (Hypothetical Document Embeddings).
Instead of embedding the raw query, an LLM first generates a hypothetical answer/document that would satisfy the query, and that hypothetical document's embedding is used for retrieval — since document-like text often embeds closer to actual relevant documents than a short question does.
48. What is query decomposition and when is it necessary?
Complex, multi-part questions are broken into simpler sub-questions, each retrieved and answered independently (or sequentially, feeding into each other), then synthesized into a final answer. Necessary for compositional/multi-hop questions that a single retrieval pass over the original query wouldn't resolve.
49. What is routing in RAG, and how does it work in multi-source systems?
A routing layer (often a lightweight LLM classifier or embedding-based classifier) decides which knowledge source, index, or retrieval strategy to use for a given query — e.g., routing a "pricing" query to a product database and a "how-to" query to documentation, improving relevance and efficiency in systems spanning multiple heterogeneous sources.
50. Explain Reciprocal Rank Fusion (RRF) for combining multiple retrieval result lists.
RRF combines rankings from multiple retrieval methods (e.g., dense + sparse) by scoring each document as the sum of 1/(k + rank) across all lists it appears in, rewarding documents that rank well across multiple methods without needing to normalize disparate similarity score scales — simple and robust for hybrid search fusion.
51. What is self-querying retrieval?
An LLM parses the natural language query to automatically extract structured filter conditions (e.g., "papers from 2023 about transformers" → semantic query "transformers" + metadata filter year=2023), combining semantic search with precise structured filtering without the user needing to specify filters explicitly.
52. How do you implement retrieval over structured data (SQL databases) combined with unstructured RAG?
Use a routing/agentic layer where the LLM determines if a query needs structured data (generates and executes SQL against the database, often called "Text-to-SQL") versus unstructured retrieval (vector search), or combines both — sometimes called hybrid or "structured RAG."
53. What is GraphRAG and when would you use a knowledge graph over standard vector retrieval?
GraphRAG builds/uses a knowledge graph of entities and relationships extracted from the corpus, enabling retrieval that follows explicit relational structure (multi-hop entity relationships) rather than just semantic similarity — valuable for questions requiring relational reasoning (e.g., "who are all the people connected to X through Y") that vector similarity alone struggles with.
54. What is iterative/recursive retrieval?
The system retrieves, generates an intermediate answer or assessment, and uses that to inform a subsequent retrieval query, repeating until sufficient information is gathered or a stopping criterion is met — useful for complex research-style questions requiring progressively refined information gathering.
55. How do you handle retrieval for conversational (multi-turn) RAG where queries depend on prior context?
Rewrite/contextualize the current turn's query using conversation history (often via an LLM call that resolves references like "it" or "that" into an explicit standalone query) before running retrieval, since raw follow-up queries in isolation often lack the context needed for accurate retrieval.
56. What is the "needle in a haystack" test and how does it relate to retrieval evaluation?
It tests whether a system can retrieve/recall a specific fact ("needle") planted within a large volume of distractor content ("haystack") at varying positions and context lengths — commonly used to evaluate both long-context LLMs' recall and RAG retrieval pipelines' ability to surface sparse relevant information.
57. What is negative/hard-negative mining and why does it matter for retrieval quality?
Hard negatives are documents that are superficially similar (high embedding similarity) but actually irrelevant/incorrect. Including them during embedding model fine-tuning (contrastive training) sharpens the model's ability to discriminate fine-grained relevance, significantly improving retrieval precision over training with only random negatives.
58. How do you handle retrieval when the answer isn't explicitly present in any single document (requires synthesis across multiple)?
Retrieve a broader set of top-k relevant chunks across potentially multiple documents, ensure the prompt explicitly instructs the LLM to synthesize across sources, and consider multi-hop or iterative retrieval to progressively gather the necessary pieces before final generation.
59. What is time-aware or recency-biased retrieval, and how do you implement it?
For domains where newer information should be preferred (news, pricing, policy), combine semantic similarity score with a recency decay factor (e.g., exponential decay by document age) in the final ranking, or apply hard metadata filters/boosts for date ranges, rather than relying on semantic similarity alone.
60. How would you design retrieval for a RAG system with strict document-level access control (multi-tenant enterprise)?
Enforce access control at the retrieval layer, not just the UI — filter the vector search itself by tenant/permission metadata (never retrieve, even into the LLM context, documents the user isn't authorized to see), and audit-log retrieval access for compliance; never rely on the LLM to "choose not to use" unauthorized content it was given.
E. Reranking & Fusion (Q61–70)
61. What is reranking and why is it added as a separate stage after initial retrieval?
Initial retrieval (dense/hybrid, often over the full corpus) prioritizes speed via cheaper bi-encoder similarity. Reranking applies a more expensive but more accurate cross-encoder (or LLM-based) model to just the small candidate set (e.g., top 50-100), re-ordering by finer-grained relevance before passing the final top-k to the LLM.
62. Compare cross-encoder rerankers vs LLM-based reranking.
Cross-encoder rerankers (e.g., Cohere Rerank, BGE-reranker) are purpose-trained, fast, and cost-effective for scoring query-document relevance. LLM-based reranking (prompting a general LLM to score/rank candidates) can incorporate more nuanced/contextual judgment and reasoning but is slower and more expensive — choice depends on latency/cost budget vs required nuance.
63. What is Maximal Marginal Relevance (MMR) and what problem does it solve?
MMR re-ranks retrieved results to balance relevance with diversity, penalizing candidates too similar to already-selected results — preventing the top-k from being near-duplicate chunks (e.g., five near-identical paragraphs from the same section) and ensuring broader coverage of distinct relevant information.
64. How does reranking improve overall RAG answer quality beyond just retrieval metrics?
By ensuring the most genuinely relevant chunks occupy the positions in context the LLM attends to most (start/end, per "lost in the middle"), and by filtering out superficially-similar-but-irrelevant chunks that would otherwise dilute or confuse generation — directly improving downstream answer accuracy, not just retrieval recall metrics.
65. What is context compression in RAG pipelines?
After retrieval (and optionally reranking), an additional step extracts/summarizes only the most relevant sentences/spans from each retrieved chunk (rather than passing full chunks), reducing token usage and noise while preserving the information needed to answer the query — useful when retrieved chunks are large but only partially relevant.
66. How do you decide the optimal top-k value to retrieve before and after reranking?
Tune empirically: retrieve a broader initial candidate set (e.g., top-50) to maximize recall cheaply, then rerank down to a smaller final set (e.g., top-5) balancing context window budget, cost, and the "lost in the middle" risk of over-including — validate via end-to-end answer quality eval, not retrieval metrics alone.
67. What is Cohere Rerank / BGE-reranker and how are they typically integrated into a pipeline?
These are purpose-built cross-encoder models exposed via API or open weights that take a (query, document) pair and output a relevance score. Integrated as a post-retrieval step: pass the top-N candidates from initial vector search through the reranker, sort by its score, and take the final top-k for the LLM prompt.
68. What are the latency/cost trade-offs of adding a reranking stage, and how do you justify it?
Reranking adds an extra model call (latency + cost) per query, but the resulting improvement in context relevance often meaningfully increases answer accuracy and reduces hallucination — justified when initial retrieval precision is a bottleneck, which you'd confirm via ablation testing (measuring answer quality with/without reranking on your eval set).
69. What is score fusion and how does it differ from rank fusion (like RRF)?
Score fusion combines raw similarity/relevance scores (often after normalization, e.g., min-max scaling) from multiple retrieval methods via weighted sum. Rank fusion (RRF) instead combines methods based on each document's rank position, sidestepping the challenge of normalizing incompatible score scales across different retrieval algorithms — RRF is generally more robust when combining heterogeneous methods.
70. How would you handle a case where reranking consistently demotes a document type that's actually important (e.g., short FAQ answers get outranked by longer docs)?
Investigate reranker bias (many cross-encoders have length bias favoring longer text), consider chunk-type-aware boosting/normalization, fine-tune or select a reranker evaluated specifically on your document type distribution, or apply post-reranking business rules ensuring minimum representation from key document categories.
F. Generation & Prompting for RAG (Q71–80)
71. How should a RAG system prompt be structured to maximize grounded, accurate answers?
Clearly separate instructions, retrieved context (with source labels), and the user query; explicitly instruct the model to answer only from the provided context and to say "I don't know" if the context is insufficient; request citations pointing to specific sources; and keep instructions concise to avoid diluting attention on the actual context.
72. How do you instruct an LLM to say "I don't know" instead of hallucinating when retrieval fails?
Explicitly prompt the model that it's acceptable and expected to state uncertainty or lack of information rather than guess, provide few-shot examples of "insufficient context" cases, and pair this with a retrieval-confidence check (e.g., low similarity scores) that can short-circuit generation entirely before the LLM is even prompted.
73. How do you implement citation generation in RAG responses?
Label each retrieved chunk with a source identifier in the prompt, instruct the model to reference sources by ID inline in its answer (e.g., "[1]"), and post-process to map those IDs back to actual document links/titles for the UI — some approaches instead do post-hoc attribution by matching generated sentences back to source chunks via NLI/similarity.
74. What is the risk of over-reliance on retrieved context vs the model's parametric knowledge, and how do you balance it?
Over-reliance purely on retrieved context can produce brittle, overly narrow answers if retrieval is imperfect; over-reliance on parametric knowledge risks outdated/hallucinated facts. Balance via prompting that prioritizes retrieved context for facts while allowing general reasoning/synthesis, and evaluation that specifically checks for contradictions between the two.
75. How do you handle conflicting information across multiple retrieved documents in generation?
Instruct the model explicitly to surface and acknowledge conflicts rather than silently picking one source, prioritize by metadata signals (recency, authority/source trust score) when available, and consider a stricter mode where genuinely conflicting critical information triggers a clarification request rather than a confident single answer.
76. What is the impact of retrieved context ordering on generation quality, and how do you optimize it?
Given the "lost in the middle" effect, placing the most relevant/highest-confidence retrieved chunks at the beginning and/or end of the context (rather than by arbitrary or purely rank order in the middle) can measurably improve the model's use of that information in its answer.
77. How do you prevent the LLM from "leaking" instructions or internal system prompt content in RAG responses?
Clearly demarcate system instructions from context/user content with structural boundaries, explicitly instruct the model not to reveal system instructions, test with adversarial prompts probing for leakage, and apply output-side filtering as a backstop guardrail.
78. What is answer synthesis across multiple documents, and what generation strategies help produce coherent multi-source answers?
Rather than treating each chunk independently, prompt the model to identify overlapping/complementary information across sources and produce a unified, non-redundant answer; techniques like map-reduce summarization (summarize each source, then synthesize summaries) help when the number of relevant sources is large.
79. How do you handle cases where the user query is a follow-up that references the previous RAG answer, not just the original documents?
Include recent conversation turns (including the assistant's prior grounded answer) in the generation context alongside newly retrieved documents, and consider whether new retrieval is even needed for a given follow-up (some follow-ups are pure clarification/reformatting of already-retrieved information).
80. What is the role of temperature/sampling settings specifically in RAG generation, and how should they differ from open-ended generation?
RAG generation typically benefits from lower temperature (more deterministic, closer to greedy decoding) since the goal is faithful grounding to retrieved facts rather than creative diversity — high temperature increases the risk of the model deviating from or embellishing beyond the provided context.
G. Evaluation of RAG Systems (Q81–90)
81. What are the key dimensions to evaluate in a RAG system?
Retrieval quality (are the right documents found — precision/recall), groundedness/faithfulness (does the answer stick to retrieved content), answer relevance (does it actually address the query), and end-to-end correctness — each requires distinct metrics since a failure in any one stage can look like a generation problem.
82. Explain the RAGAS evaluation framework and its core metrics.
RAGAS provides LLM-based reference-free metrics: faithfulness (are claims in the answer supported by retrieved context), answer relevance (does the answer address the query), context precision (are retrieved chunks actually relevant, ranked appropriately), and context recall (was all necessary information retrieved) — enabling automated RAG evaluation without needing large hand-labeled ground truth sets.
83. How do you evaluate retrieval quality independent of generation quality?
Use a labeled eval set of (query, relevant document/chunk) pairs and measure standard IR metrics: Recall@k (is the relevant doc in the top-k), Precision@k, Mean Reciprocal Rank (MRR), and NDCG (accounting for ranking order) — isolating retrieval evaluation helps pinpoint whether failures originate in retrieval or generation.
84. What is context precision and context recall in RAG evaluation, and why measure both?
Context recall measures whether all necessary relevant information was retrieved (missing it caps the best possible answer quality). Context precision measures how much of the retrieved context was actually relevant/useful (low precision wastes context budget and can dilute/confuse generation) — a system can be strong on one and weak on the other.
85. How do you build a labeled evaluation dataset for RAG when you don't have pre-existing ground truth?
Sample representative real (or synthetic, LLM-generated) queries, have domain experts (or a strong LLM with human verification) identify the ground-truth relevant document(s) and ideal answer for each, and periodically expand the set with production failure cases discovered through monitoring.
86. What is answer faithfulness/groundedness and how is it typically measured automatically?
Faithfulness measures whether every factual claim in the generated answer is entailed by the retrieved context (not the model's external/parametric knowledge). Measured by decomposing the answer into atomic claims and checking each against the context via NLI models or an LLM-judge prompted specifically for entailment checking.
87. How do you evaluate a RAG system's handling of "unanswerable" queries (where no relevant context exists)?
Include deliberately unanswerable queries in your eval set and measure whether the system correctly abstains/states uncertainty rather than hallucinating a confident-sounding but ungrounded answer — this is a critical, often-overlooked failure mode distinct from standard answer-quality metrics.
88. What is the role of human evaluation in RAG systems, and when is it indispensable over automated metrics?
Human eval remains indispensable for judging nuanced answer quality (tone, completeness, subtle factual errors automated NLI checks miss), calibrating/validating automated LLM-judge metrics periodically, and evaluating on genuinely novel or ambiguous production queries not covered by static eval sets.
89. How do you set up continuous evaluation/regression testing for a RAG system in production?
Maintain a golden eval set covering key query types and known edge cases, run it automatically on every pipeline change (chunking strategy, embedding model, prompt, reranker), track metric trends over time (not just pass/fail thresholds), and periodically refresh the eval set with real production failure cases surfaced by monitoring.
90. How do you diagnose whether a RAG failure is caused by retrieval or generation?
Manually inspect the actual retrieved context for the failing query: if the correct information wasn't retrieved at all, it's a retrieval failure (fix chunking/embedding/query rewriting); if the correct information was present in context but the answer is still wrong, it's a generation/faithfulness failure (fix prompting or consider a different generator model).
H. Production RAG & Scaling (Q91–100)
91. How do you scale a RAG system to handle millions of documents and high query throughput?
Use a distributed vector database with sharding/replication, ANN indexing (HNSW/IVF) tuned for the recall/latency trade-off at scale, caching for frequent queries, asynchronous/batched ingestion pipelines, and horizontal scaling of the retrieval and generation service layers independently since they have different resource profiles.
92. What is the ingestion pipeline architecture for keeping a RAG index continuously up to date?
An event-driven or scheduled pipeline that detects source document changes (webhooks, polling, or CDC from a source system), triggers re-chunking/re-embedding only for changed documents, and atomically updates the vector index — designed for incremental updates rather than full reprocessing to keep latency and cost manageable.
93. How do you handle RAG system latency budgets when multiple stages (query rewrite, retrieval, rerank, generation) each add time?
Profile and budget latency per stage against your SLA, parallelize independent steps where possible (e.g., hybrid dense+sparse retrieval concurrently), use faster/smaller models for lower-value stages (query rewriting can use a small model), cache aggressively, and consider streaming the final generation to improve perceived latency even if total time is unchanged.
94. What is the cost breakdown of a typical production RAG system, and where do costs typically concentrate?
Costs concentrate in: embedding generation (especially at ingestion of large corpora), vector database hosting/compute, reranking API calls, and LLM generation tokens (both input context and output) — generation token cost from large context windows is often the single largest recurring cost driver at scale.
95. How would you architect a RAG system to support real-time/streaming data sources (e.g., live chat logs, news feeds)?
Use a streaming ingestion pipeline (e.g., Kafka-fed) that chunks/embeds/indexes documents near-real-time, apply short TTLs or recency boosting in retrieval ranking, and separate "hot" (recent, fast-changing) from "cold" (stable, archival) indices if freshness and query patterns differ significantly between them.
96. What security/compliance considerations are unique to enterprise RAG systems?
Document-level access control enforced at the retrieval layer (not just UI), PII detection/redaction in ingested content, audit logging of what content was retrieved/shown to which user, data residency compliance for vector storage, and ensuring the LLM provider's data retention policy meets contractual/regulatory requirements.
97. How do you handle RAG for extremely large individual documents (e.g., 500-page technical manuals)?
Use hierarchical chunking/summarization (chapter → section → paragraph summaries feeding a navigable tree), parent-child retrieval to keep precise chunk-level matching while providing broader context on retrieval, and consider a table-of-contents/routing step to first narrow down the relevant section before fine-grained retrieval within it.
98. What are common failure modes you'd look for when debugging a production RAG system with declining quality?
Silent embedding/reranker model drift or version mismatch, stale index (ingestion pipeline broken), chunking regressions from a document parser update, prompt template changes, context window overflow silently truncating retrieved content, and shifts in the production query distribution away from what the system was tuned for.
99. How do you A/B test changes to a RAG pipeline (e.g., new chunking strategy or reranker) safely in production?
Run the new configuration on a shadow/canary traffic split, compare against the golden eval set metrics offline first, then compare real user engagement/satisfaction signals between control and treatment groups at small scale before full rollout, with automated rollback triggers if key quality metrics regress.
100. Design a RAG system for a domain with extremely high accuracy requirements (e.g., legal or medical) — what additional safeguards would you add?
Mandatory citation-to-source for every claim with UI links to original documents, stricter groundedness thresholds that trigger abstention over guessing, human-in-the-loop review for high-stakes outputs, retrieval from only vetted/authoritative sources with clear provenance, extensive domain-expert-curated eval sets, and conservative low-temperature generation with explicit confidence signaling in the response.
Part 3 — Model Context Protocol / MCP (100 Q&A)
A. MCP Fundamentals (Q1–15)
1. What is the Model Context Protocol (MCP)?
MCP is an open standard (introduced by Anthropic) that defines a common protocol for connecting LLM applications ("hosts") to external tools, data sources, and systems ("servers"), analogous to how USB-C standardized device connectivity — instead of every application building custom, one-off integrations for every tool.
2. What problem does MCP solve that existed before it?
Before MCP, every LLM application had to build bespoke integration code for each external tool/data source it wanted to use (an "M×N" integration problem: M applications × N tools = M×N custom integrations). MCP standardizes this into an "M+N" problem — each tool builds one MCP server, each application builds one MCP client, and they interoperate.
3. What are the three core primitives MCP servers expose?
Tools (executable functions the model can invoke, like an API call), Resources (structured/unstructured data the application can read and include as context, like files or database records), and Prompts (reusable, parameterized prompt templates the server provides for common interactions).
4. Explain the MCP host-client-server architecture.
The Host is the LLM application (e.g., Claude Desktop, an IDE, a custom agent) that manages the overall interaction and user-facing experience. It runs one or more Clients, each maintaining a dedicated 1:1 connection to a Server, which exposes tools/resources/prompts. A single host can connect to many servers simultaneously via separate clients.
5. What transport protocols does MCP support?
Primarily stdio (standard input/output, for local processes — simple, low-latency, no network exposure) and HTTP with Server-Sent Events / Streamable HTTP (for remote servers accessed over a network), allowing MCP servers to run either as local subprocesses or as remotely hosted services.
6. What underlying protocol/format does MCP use for messages?
MCP uses JSON-RPC 2.0 as its message format, defining structured requests, responses, and notifications between client and server, giving it a well-established, language-agnostic wire format with clear semantics for request/response correlation and error handling.
7. How is MCP different from a traditional REST API integration?
A REST API requires the LLM application developer to manually define, describe, and wire up each endpoint as a tool for the model. MCP servers self-describe their available tools/resources/prompts via a standardized discovery mechanism, so any MCP-compatible client can dynamically discover and use them without custom per-tool integration code.
8. How is MCP different from OpenAI-style function calling / tool use?
Function calling is a model capability (the LLM outputs structured calls); MCP is a broader protocol standardizing how those callable tools (and resources, prompts) are discovered, described, and connected across any application and any tool provider — MCP servers can be used to supply the actual tool definitions/execution behind a function-calling-capable model, they're complementary, not competing layers.
9. What is the significance of MCP being an "open standard" rather than a proprietary API?
It enables an ecosystem where any vendor can build an MCP server once and have it work across any MCP-compatible host/application, rather than needing bespoke integrations per LLM provider — reducing duplicated engineering effort industry-wide and enabling network effects similar to how HTTP or USB standardized their respective domains.
10. What was MCP built on top of / inspired by, conceptually?
It draws conceptually from the Language Server Protocol (LSP), which standardized how code editors communicate with language-specific tooling (autocomplete, diagnostics) — MCP applies a similar "one protocol, many implementations" philosophy to LLM-tool connectivity.
11. What is capability negotiation in MCP, and why does it matter?
During the initialization handshake, client and server exchange information about which protocol features/capabilities each supports (e.g., does the server support resource subscriptions, does the client support sampling), allowing both sides to adapt behavior and avoid errors from assuming unsupported features are available.
12. What is the lifecycle of an MCP connection?
Initialization (client and server exchange protocol version and capabilities) → Operation (client sends requests like tools/list, tools/call, resources/read; server can send notifications) → Shutdown (clean termination of the connection) — a well-defined lifecycle ensures predictable behavior across implementations.
13. Can an MCP host connect to multiple servers simultaneously, and how does it manage that?
Yes — a host typically instantiates one client per server it wants to connect to, each maintaining its own isolated session/connection. The host aggregates the tools/resources/prompts exposed across all connected servers to present a unified set of capabilities to the underlying LLM.
14. What is the difference between an MCP "server" and an MCP "client" in plain terms?
The server is the provider — it wraps some capability (a database, an API, a filesystem, a SaaS tool) and exposes it in MCP's standard format. The client is the consumer embedded inside the host application — it connects to a server, discovers what it offers, and relays those capabilities to the LLM and the application.
15. Why would a company build an MCP server for their product instead of just publishing API documentation for developers to build integrations from?
An MCP server makes their product immediately usable by any MCP-compatible AI application with zero custom integration work by the AI app developer — turning API documentation (which requires bespoke code per consumer) into a plug-and-play capability, significantly lowering the barrier for their product to be adopted inside AI-driven workflows.
B. Architecture & Protocol Details (Q16–30)
16. Explain the structure of a "Tool" definition in MCP.
A tool definition includes a unique name, a natural-language description (critical, since the LLM uses this to decide when/how to invoke it), and an input schema (typically JSON Schema) defining expected parameters — the server returns this via a tools/list request, and the client relays it to the LLM as an available function.
17. How does tool invocation actually work end-to-end in MCP?
The LLM (within the host) decides to call a tool based on its description and the current context, the host's client sends a tools/call JSON-RPC request with the tool name and arguments to the appropriate server, the server executes the underlying logic and returns a result (or error), which is relayed back into the LLM's context to continue generation.
18. What is a "Resource" in MCP and how does it differ from a "Tool"?
A Resource represents readable data/content (a file, a database row, an API response) identified by a URI, meant to be included as context rather than actively executed — unlike Tools, which represent actions/functions with side effects or computation. Resources are typically read via a resources/read request.
19. What is Resource templating / parameterized resources in MCP?
Servers can expose resource URI templates (e.g., file:///logs/{date}.log) rather than only static, enumerable resources, letting clients construct specific resource URIs dynamically based on parameters, useful for large or dynamically-generated data spaces that can't be feasibly listed exhaustively.
20. What are "Prompts" in MCP and what's their purpose?
Prompts are server-defined, reusable prompt templates (often parameterized) that encapsulate a well-crafted way to accomplish a specific task with that server's data/tools — surfaced to users typically as slash-commands or quick-actions in the host UI, letting server authors codify best-practice interaction patterns rather than leaving prompt engineering entirely to the end user.
21. What is "Sampling" in MCP and why is it a notable/unusual capability?
Sampling allows an MCP server to request that the host's LLM generate a completion on the server's behalf (with user permission/oversight), effectively letting a server "borrow" the host's model for its own internal reasoning needs — notable because it inverts the typical direction of control, letting servers leverage AI capability without needing their own model access/API key.
22. How does MCP handle authentication and authorization for remote servers?
For remote (HTTP-based) MCP servers, MCP has adopted OAuth 2.1-based authorization flows, allowing servers to require and validate user authentication/consent before granting access to tools and resources — critical for servers that act on behalf of a user's account (e.g., a Gmail or Slack MCP server).
23. What is the role of JSON Schema in MCP tool definitions?
JSON Schema formally specifies the expected structure, types, and constraints of a tool's input parameters, enabling both the LLM (to know how to format a valid call) and the client/host (to validate calls before sending them) to interact reliably and catch malformed requests early.
24. How does MCP support notifications and streaming updates (e.g., a resource changing)?
MCP supports server-to-client notifications (e.g., notifications/resources/updated) that inform the client when underlying data changes, and clients can subscribe to specific resources for live updates — enabling reactive UIs and agents that respond to changing state rather than only polling.
25. What is the difference between stdio transport and HTTP/SSE (or Streamable HTTP) transport in MCP, and when do you use each?
stdio is used for local servers spawned as a subprocess by the host (simple, no network/auth complexity needed, but limited to same-machine use). HTTP-based transport is used for remote servers accessed over a network, requiring proper authentication and supporting multiple concurrent clients — chosen based on whether the tool/data lives locally or needs to be accessed as a hosted service.
26. What is the purpose of the "initialize" handshake in the MCP protocol?
It's the first exchange in an MCP session where client and server agree on the protocol version to use and declare their respective supported capabilities (e.g., resource subscriptions, sampling support), preventing version mismatches or capability assumption errors later in the session.
27. How does error handling work in MCP tool calls?
Tool execution errors are returned as part of the tool result (often with an isError flag and descriptive content) rather than as protocol-level JSON-RPC errors when the failure is domain-specific (e.g., "file not found"), allowing the LLM to see and potentially reason about/recover from the error within the conversation, while true protocol-level errors use standard JSON-RPC error responses.
28. Can MCP servers expose dynamically changing sets of tools, and how does the protocol support this?
Yes — servers can send a tools/list_changed notification when their available tools change (e.g., after connecting to a new backend), prompting the client to re-fetch the updated tool list, supporting scenarios where tool availability depends on runtime state rather than being fixed at connection time.
29. What is the significance of tool/resource descriptions being natural language, and what happens if they're poorly written?
Since the LLM relies entirely on the natural language description (not the code) to decide when and how to use a tool, vague, ambiguous, or missing descriptions directly cause incorrect tool selection or malformed calls — well-crafted, specific descriptions with examples are one of the highest-leverage things an MCP server author can do for reliability.
30. How does MCP's design address the problem of context window bloat when a host is connected to many servers with many tools?
This remains an active challenge; approaches include on-demand/lazy tool discovery rather than loading every tool description upfront, semantic tool search/filtering to surface only relevant tools per query, and namespacing/grouping tools by server so hosts can selectively enable only needed servers per session rather than always including all connected servers' full tool sets.
C. Building MCP Servers (Q31–45)
31. What are the main steps to build a basic MCP server?
Choose an SDK (Python, TypeScript, etc.), define the server's tools/resources/prompts with clear schemas and descriptions, implement the underlying handler logic connecting to the actual system (API/database/filesystem), choose a transport (stdio for local, HTTP for remote), and test it against an MCP-compatible client/host.
32. What SDKs/languages are officially supported for building MCP servers?
Official SDKs exist for Python, TypeScript/JavaScript, Java, Kotlin, and C#, among others, maintained by Anthropic and the broader open-source community, each providing the JSON-RPC protocol handling, transport implementations, and higher-level decorators/abstractions for defining tools, resources, and prompts.
33. How do you design good tool descriptions for an MCP server?
Be explicit and specific about what the tool does, when to use it (and when not to), the exact expected format/units of parameters, what the response looks like, and edge cases/limitations — write descriptions as if instructing a competent but context-free assistant, since that's effectively what the LLM is.
34. What is the best practice for handling sensitive operations (e.g., deleting data, sending emails) in an MCP server?
Require explicit confirmation flows where feasible (returning a preview/dry-run result before an irreversible action executes), design tools with the principle of least privilege (narrow, specific tools rather than broad "do anything" tools), implement server-side authorization checks independent of what the LLM claims, and log all state-changing operations for auditability.
35. How should you structure input schemas to minimize malformed tool calls from the LLM?
Keep parameter names self-descriptive, use enums/constrained types where possible rather than free text, provide sensible defaults, avoid deeply nested or overly complex schemas, include examples in the description, and validate inputs server-side rather than trusting the LLM's output is always well-formed.
36. How do you version an MCP server's API without breaking existing clients?
Follow semantic versioning for the server itself, avoid breaking changes to existing tool names/schemas (add new tools/parameters rather than mutating existing ones), use the protocol version negotiation during initialization to gracefully handle capability differences, and clearly document deprecations with a migration path/timeline.
37. What testing strategies are important for MCP servers before production release?
Unit test each tool handler's business logic independently of the protocol layer, use the official MCP Inspector tool for interactive manual testing of the protocol-level behavior, write integration tests that simulate realistic multi-turn LLM tool-use sequences, and test error paths/malformed input handling explicitly.
38. How would you design an MCP server that wraps a large, complex API (e.g., a full CRM system) without overwhelming the LLM with too many tools?
Group related operations into a smaller number of well-designed, higher-level tools rather than exposing every raw API endpoint 1:1, use resource templates for read-heavy data browsing instead of many separate "get" tools, and consider progressive disclosure (a "search/list" tool that then informs which detail tool to call) rather than flat, exhaustive tool lists.
39. What is the MCP Inspector and how is it used in development?
It's an official interactive developer tool that connects directly to an MCP server (without needing a full LLM host) to manually list/invoke tools, browse resources, and test prompts — used during development to debug and validate server behavior in isolation before integrating with a real LLM application.
40. How do you handle long-running operations in an MCP tool call (e.g., a job that takes minutes to complete)?
Rather than blocking the synchronous tool call, design an async pattern: the tool call kicks off the job and immediately returns a job ID/status, with a separate "check status" tool (or resource subscription/notification) the LLM can poll or be notified on, avoiding protocol-level timeouts on long-running work.
41. How should an MCP server handle rate limiting from an underlying third-party API it wraps?
Implement server-side rate limit tracking/backoff against the upstream API, return clear, actionable error messages to the calling LLM when limited (so it can inform the user or retry appropriately rather than looping blindly), and consider caching frequent read-only requests to reduce upstream call volume.
42. What are best practices for logging and observability in an MCP server?
Log every tool invocation with parameters (redacting sensitive data), execution time, and outcome (success/error); correlate logs with session/request IDs for tracing multi-step agent interactions; expose metrics (call volume, latency, error rate per tool) for monitoring; and avoid logging full sensitive payloads (credentials, PII) in plaintext.
43. How do you design an MCP server to be stateless vs stateful, and what are the trade-offs?
Stateless servers (each call self-contained, no server-side session memory) are simpler to scale horizontally and reason about, but push more burden onto the client/host to manage context. Stateful servers can offer richer, more efficient multi-step interactions (e.g., maintaining an open file handle or transaction) but require careful session lifecycle and cleanup management, and complicate horizontal scaling.
44. How would you implement pagination for a resource or tool that could return very large result sets in MCP?
Support cursor-based or offset-based pagination parameters in the tool/resource request, return a continuation token in the response when more results exist, and document clearly in the tool description that results may be paginated so the LLM knows to request subsequent pages when needed rather than assuming a single call returns everything.
45. What are common mistakes developers make when first building MCP servers?
Writing vague/generic tool descriptions the LLM can't reliably act on, exposing too many overly granular tools instead of a few well-designed ones, not validating/sanitizing inputs server-side (trusting the LLM's output blindly), ignoring authorization/access control at the tool-execution layer, and not testing with actual multi-turn LLM interactions before shipping.
D. MCP Clients & Hosts (Q46–55)
46. What responsibilities does an MCP host application have beyond just running clients?
Managing the overall user experience and conversation flow, aggregating capabilities across multiple connected servers into what's presented to the LLM, enforcing user consent/permission for tool calls and data access, managing the underlying LLM's context window budget across potentially many tool/resource results, and handling error/fallback UX when servers are unavailable.
47. How does a host decide which tools to actually expose to the LLM in a given conversation turn, especially with many connected servers?
Approaches range from exposing everything from all connected/enabled servers (simplest, but risks context bloat and tool-selection confusion), to semantic filtering (retrieving only tools relevant to the current query via embedding similarity), to explicit user/developer configuration of which servers are active per session or workspace.
48. What is the role of user consent in the MCP architecture, and why is it emphasized in the spec?
Because tools can perform real-world actions (sending emails, modifying files, spending money) and resources can expose potentially sensitive data, the MCP spec emphasizes that hosts should obtain explicit user consent before invoking tools or granting a server access to data, treating the user as the ultimate authority over what the AI is permitted to do on their behalf.
49. How should a host handle a scenario where a connected MCP server becomes unavailable mid-session?
Gracefully degrade rather than crash — inform the user/LLM that the server/its tools are currently unavailable, remove or mark unavailable tools from what's offered to the LLM to prevent it from attempting calls that will fail, and implement reconnection logic with appropriate backoff for transient failures.
50. What is the difference between a "local" MCP client-server connection and a "remote" one, from the host's perspective?
Local connections (stdio) mean the host spawns and manages the server as a subprocess on the same machine, with implicit trust and no network auth needed. Remote connections (HTTP) mean the host connects to a server potentially operated by a third party over the network, requiring proper authentication/authorization and treating the server as a less-trusted external dependency.
51. How do popular AI applications (e.g., Claude Desktop, IDEs) implement MCP hosting in practice?
They typically provide a configuration mechanism (e.g., a config file or UI) for users to register MCP servers they want connected, spin up clients for each on startup, surface available tools/prompts/resources in the chat/coding interface, and mediate all tool-call approvals through their existing UI patterns for user confirmation.
52. What UX patterns are important for hosts to implement around tool call approval?
Clearly show the user what tool is being called with what parameters before/as execution happens (not just after), allow granular approval (per-call, per-tool, or "always allow" for trusted low-risk tools), make destructive/irreversible actions require explicit extra confirmation, and provide clear visibility into what data was read/sent to which server.
53. How does context window management work when a host is aggregating resources/tool results from multiple MCP servers?
The host must budget the limited context window across the system prompt, conversation history, and all tool/resource content being injected — strategies include summarizing/truncating large tool results, prioritizing more relevant recent results, and giving the LLM/agent logic to selectively request more detail only when needed rather than dumping everything upfront.
54. What is the trade-off a host developer faces between exposing many MCP servers/tools vs curating a smaller, focused set?
More servers/tools increase the range of tasks the assistant can accomplish but increase context window usage, latency, cost, and the risk of the LLM selecting the wrong or a suboptimal tool among many similar options — curation (enabling only relevant servers per workspace/task) generally improves reliability at the cost of requiring more upfront configuration.
55. How would you design a host application that needs to support both MCP tools and traditional hardcoded function-calling tools simultaneously?
Normalize both into a common internal tool representation/interface at the application layer so the LLM-facing tool list is unified regardless of source, route execution to the appropriate handler (MCP client call vs direct function invocation) based on tool origin, and ensure consistent error handling/UX (consent, logging) is applied uniformly across both.
E. Tools, Resources & Prompts Deep Dive (Q56–70)
56. What makes a "well-designed" MCP tool from an LLM-usability perspective?
It has a single, clear responsibility (not an overloaded multi-purpose function), a name and description unambiguous enough that the LLM rarely confuses it with a similar tool, a minimal but sufficient parameter set, predictable and well-structured output, and explicit documentation of failure modes/edge cases in the description.
57. When should functionality be exposed as a Tool vs a Resource in MCP?
Use a Tool when the operation performs an action, computation, or has side effects (searching, sending, creating, updating). Use a Resource when it's about surfacing readable data/content for context (a file's contents, a database record) that the model should be able to read but isn't "invoking" as an action.
58. How do Prompts in MCP differ from simply instructing the LLM via a system prompt?
MCP Prompts are structured, discoverable, server-provided templates (often surfaced as explicit user-invokable commands, like a slash-command) tied to that server's specific domain/data — they codify expert-crafted interaction patterns that any user of the server can invoke consistently, rather than relying on ad hoc system prompt instructions the host developer writes independently.
59. Can MCP Resources include binary data (images, PDFs), and how is that handled?
Yes — resources can return content typed appropriately (e.g., base64-encoded binary with a MIME type) alongside or instead of plain text, allowing servers to expose non-text content like images or documents that the host/LLM (if multimodal) can process.
60. What is resource subscription in MCP and what use cases does it enable?
A client can subscribe to a specific resource to receive notifications when its content changes, enabling use cases like a live-updating dashboard, a file being actively edited elsewhere, or a monitoring feed — without needing to inefficiently poll the resource repeatedly for changes.
61. How would you design a set of MCP tools for a project management tool (like Jira/Asana) integration?
Likely tools: search/list issues (with filters), get issue details, create issue, update issue status/fields, add comment — each narrowly scoped with clear parameters; resources could expose read-heavy views like "my open tickets"; a prompt template might codify a common workflow like "triage my backlog."
62. What is the risk of tool name collisions when a host connects to multiple MCP servers, and how is it handled?
Two servers could expose tools with the same name (e.g., both a Gmail and Outlook server offering "send_email"), causing ambiguity for the LLM/host. This is typically handled via namespacing — prefixing tool names with the server identifier internally — so the LLM sees clearly disambiguated options.
63. How should an MCP tool's output be structured to be maximally useful to the LLM for follow-up reasoning?
Structured, consistent formatting (not raw dumps of unprocessed API responses), inclusion of IDs/references needed for potential follow-up tool calls, human-readable summaries alongside raw data where relevant, and clear error/status signaling — output should be designed for the LLM to parse and reason over, not just for a human reading logs.
64. What is the significance of a tool being marked as "read-only" vs having side effects, in terms of host/UX behavior?
Read-only tools (pure data retrieval) are generally lower-risk and can often be auto-approved or run without explicit per-call confirmation, whereas tools with side effects (writes, sends, deletes) typically warrant explicit user consent/confirmation each time or per-session opt-in — this distinction is important for building trustworthy, low-friction agent UX without exposing users to unwanted actions.
65. How would you handle a tool that requires multi-step confirmation (e.g., "search for a flight" then "book the selected flight")?
Design them as separate, distinct tools rather than one combined tool — a search tool returns options/IDs, and a separate booking tool takes a specific option ID as a parameter, naturally creating a checkpoint where the user/host can review and approve before the consequential action (booking) executes.
66. Can MCP Prompts accept parameters, and how does that work in practice?
Yes — prompt templates can define expected arguments (similar to tool input schemas), and when a user/host invokes the prompt, they supply values that get interpolated into the resulting message(s) sent to the LLM, letting server authors create flexible, reusable interaction templates rather than only static text.
67. How do you handle a tool whose behavior should differ based on user permissions/role (e.g., an admin vs regular user)?
Enforce the actual authorization check server-side, based on the authenticated user's identity/token passed with the request (not on any instruction from the LLM) — the tool's description can note that certain operations require elevated permissions, but the server must be the source of truth, returning an appropriate permission-denied error if the check fails.
68. What is the best practice for exposing search/list-style tools (e.g., searching a large document set) via MCP?
Support query parameters (filters, sorting, pagination), return a manageable, summarized result set rather than full content by default (with a separate "get details" tool/resource for full content on a specific item), and document expected query syntax/capabilities clearly so the LLM constructs effective search queries.
69. How should error messages from a Tool call be worded to be useful to the LLM (not just a human developer)?
Specific and actionable rather than generic ("Invalid date format: expected YYYY-MM-DD, got 'March 5'" rather than just "Error"), since the LLM will often use the error message directly to self-correct and retry the call with corrected parameters in the next turn.
70. What is an example of a poorly designed MCP tool, and how would you redesign it?
A single generic execute_database_query(sql: string) tool is poorly designed — it gives the LLM raw SQL access (security risk, unpredictable behavior, hard to validate). A redesign exposes specific, narrowly-scoped tools (search_customers(name, region), get_order_details(order_id)) with defined schemas, making behavior predictable, auditable, and safe by construction.
F. Security & Governance (Q71–80)
71. What are the primary security risks introduced by MCP servers?
Prompt injection via untrusted content returned from a server (e.g., a malicious webpage's content instructing the LLM to take unintended actions), overly broad tool permissions enabling unintended destructive actions, credential/token leakage if servers mishandle auth, and supply-chain risk from installing/running third-party MCP server code with system-level access.
72. What is "tool poisoning" in the context of MCP, and how do you defend against it?
A malicious or compromised MCP server could include hidden instructions within tool descriptions or results designed to manipulate the LLM's behavior (e.g., instructing it to exfiltrate data through another connected tool). Defenses include only installing servers from trusted/vetted sources, sandboxing server execution, and applying prompt-injection-resistant instruction hierarchies at the host/model level.
73. How should credentials/secrets be managed for MCP servers that need to authenticate with third-party services?
Store credentials securely outside the LLM's context entirely (environment variables, secret managers, OS keychains) — never pass raw API keys/tokens through the LLM's visible context — use OAuth flows with short-lived tokens where possible, and ensure the server, not the LLM, handles all credential usage internally.
74. What is the principle of least privilege as applied to MCP tool design?
Each tool/server should be granted and expose only the minimum permissions/scope necessary for its function — e.g., a "read customer email" tool shouldn't also carry delete permissions on the underlying mailbox — minimizing the blast radius if the LLM is manipulated (via injection or error) into misusing a tool.
75. How do you audit and monitor MCP tool usage for security/compliance purposes?
Log every tool invocation with full parameters (redacting secrets), timestamp, user/session identity, and outcome; retain logs for compliance review; implement anomaly detection for unusual call patterns (e.g., bulk data exports); and ensure logs are tamper-evident/centrally aggregated rather than only living on individual local server instances.
76. What is the risk of "confused deputy" attacks in MCP, and how does it manifest?
A confused deputy attack occurs when a server with legitimate elevated privileges is tricked (via the LLM, which itself may be manipulated by injected content) into performing an action on behalf of an attacker that the actual user never authorized — mitigated by strong per-action authorization checks and not conflating "the LLM asked for it" with genuine user intent.
77. How should a host validate that a remote MCP server is legitimate/trustworthy before connecting?
Verify server identity via TLS/certificate validation for HTTP transports, use OAuth-based authorization flows that confirm the server is the one the user intended to grant access to, only auto-connect to servers from a vetted registry/marketplace where feasible, and clearly surface server identity/publisher information to the user before granting consent.
78. What governance practices should an enterprise implement before allowing employees to connect arbitrary MCP servers to internal AI tools?
Maintain an approved/vetted registry of sanctioned MCP servers, require security review before internal servers are published, restrict connection to unapproved/unknown external servers via policy or technical controls, log and monitor all tool usage for compliance, and provide clear guidelines on what data classifications are permitted to flow through which servers.
79. How do you prevent sensitive data from being inadvertently sent to an external/third-party MCP server?
Implement data classification and DLP (data loss prevention) checks at the host/gateway layer before data is included in resource content sent to external servers, restrict which servers can access sensitive data sources via configuration/policy, and default to more restrictive/local-only servers for highly sensitive internal data.
80. What is the difference between securing an MCP server itself vs securing the broader MCP-enabled agent system it's part of?
Securing the server means hardening its own code (input validation, auth, least privilege) against direct attacks. Securing the broader system additionally requires defending against prompt injection propagating through the LLM's reasoning across multiple tools/servers, ensuring consent/authorization flows can't be bypassed by manipulated model behavior, and monitoring emergent risks from tool composition (e.g., chaining a "read" and a "send" tool to exfiltrate data) that no single server's security review would catch alone.
G. MCP vs Alternatives & Ecosystem (Q81–90)
81. How does MCP compare to LangChain tools/agents as an approach to giving LLMs external capabilities?
LangChain tools are a framework-specific abstraction tightly coupled to LangChain's own agent/orchestration code, requiring custom integration per tool within that framework. MCP is a protocol-level standard independent of any specific orchestration framework, meaning a single MCP server implementation works across any MCP-compatible host, not just one framework's ecosystem.
82. How does MCP compare to OpenAI's "GPTs"/plugins model?
OpenAI's plugin/GPTs actions model is a proprietary, provider-specific mechanism for extending ChatGPT specifically. MCP is an open, vendor-neutral protocol designed to work across any compatible LLM application, not tied to a single provider's ecosystem — aiming for broader interoperability rather than a single-platform extension mechanism.
83. Is MCP a replacement for traditional API integration platforms (like Zapier or MuleSoft), or complementary?
Largely complementary — MCP standardizes how an LLM discovers and invokes capabilities in an AI-native, protocol-first way optimized for LLM reasoning (natural language descriptions, dynamic discovery), while integration platforms like Zapier focus on pre-built, often non-AI, workflow automation between services; some platforms are themselves building MCP servers to expose their existing integrations to AI agents.
84. What role does the MCP server registry/directory ecosystem play?
Public registries/directories (analogous to a package registry like npm) let developers discover existing MCP servers for common tools/services rather than building from scratch, and let host applications offer curated, one-click connection experiences to users — accelerating ecosystem adoption similar to how app stores accelerated mobile app distribution.
85. How does MCP relate to the broader trend of "agentic AI" and multi-step autonomous workflows?
MCP provides the standardized plumbing (tool/data access) that agentic systems need to actually act in the world across many different systems, without which every agent framework would need bespoke per-tool integration code — it's an enabling infrastructure layer for the broader shift toward LLMs that plan and execute multi-step tasks using external capabilities.
86. What are the current limitations/immaturities of the MCP ecosystem as of its current state?
Areas still maturing include standardized solutions for context-window-efficient tool discovery at scale (many connected servers), mature security/trust tooling for third-party server vetting, consistent authorization UX patterns across different host implementations, and broad tooling for observability/debugging across multi-server agent sessions.
87. How might MCP evolve to better support very large numbers of connected tools/servers in a single session?
Likely directions include semantic/dynamic tool discovery (only surfacing relevant tools per query rather than the full static list), hierarchical or namespaced tool organization, and richer capability negotiation letting hosts fetch tool details lazily rather than upfront — active areas of community and specification development.
88. How would you evaluate whether to build a custom MCP server vs use an existing one from the ecosystem for a given integration need?
Search existing registries/directories first for a maintained, well-reviewed server matching your needs (avoiding duplicated effort and benefiting from community-vetted security/quality); build custom when you need proprietary internal system access, tighter control over tool design for your specific use case, or when no adequately maintained option exists.
89. What is the relationship between MCP and vector databases/RAG systems — are they competing or complementary?
Complementary — MCP is a general protocol for connecting to any tool or data source, including potentially a vector database/RAG retrieval system exposed as an MCP server (e.g., a "search_knowledge_base" tool backed by a RAG pipeline). MCP doesn't replace RAG's retrieval techniques; it can standardize how an agent accesses a RAG system alongside other tools.
90. Why might an organization choose to expose internal RAG search as an MCP tool rather than embedding RAG logic directly into their LLM application code?
Exposing it as an MCP server makes the RAG capability reusable and directly accessible from any MCP-compatible host/agent (not just one specific application), decouples the RAG implementation from any single application's codebase, and allows independent versioning/improvement of the retrieval system without requiring changes in every consuming application.
H. Advanced MCP & Production Deployment (Q91–100)
91. How would you architect a production deployment of an MCP server that needs high availability and horizontal scaling?
Deploy as a stateless HTTP-based service behind a load balancer with multiple replicas, externalize any session state to a shared store (e.g., Redis) rather than in-process memory, implement health checks for orchestration (Kubernetes) to manage instance lifecycle, and ensure idempotency for retried requests given network-layer failures.
92. What observability/monitoring should be in place for an MCP server running in production?
Per-tool invocation metrics (latency, error rate, call volume), distributed tracing correlating tool calls across a multi-step agent session, alerting on elevated error rates or latency degradation, and structured logs enabling post-incident debugging of exactly what a given agent session did and why.
93. How do you handle backward compatibility when evolving an MCP server's tool schemas over time?
Add new optional parameters rather than changing existing required ones, avoid renaming or removing existing tools abruptly (deprecate with warning periods and clear migration guidance instead), version the server/protocol capabilities explicitly, and maintain integration tests against previous schema versions during a transition period.
94. What deployment patterns exist for making an MCP server available to end users within an enterprise (vs a single developer's local setup)?
Common patterns: centrally hosted remote MCP servers behind enterprise SSO/OAuth that any employee's approved AI application can connect to, or centrally distributed/managed local server configurations pushed via IT device management for stdio-based servers — both aim to avoid every employee independently installing/configuring servers with inconsistent security posture.
95. How would you design load testing for an MCP server expected to handle high concurrent agent traffic?
Simulate realistic multi-turn agentic call patterns (not just isolated single tool calls, since agents often chain multiple calls per task), test under concurrent session load reflecting expected production concurrency, measure both latency and correctness under load (not just throughput), and specifically test behavior under upstream dependency degradation (rate limits, timeouts from wrapped APIs).
96. What is the operational difference between debugging a traditional API integration failure vs debugging an MCP-based agent workflow failure?
Traditional API failures are typically deterministic and reproducible from logs alone. MCP-based agent failures often require reconstructing the LLM's reasoning trace (why did it choose this tool, with these parameters, at this point) alongside the tool execution logs, since the failure may originate in the model's tool-selection/argument-construction logic rather than the tool implementation itself.
97. How do you handle graceful degradation in a host application when one of several connected MCP servers is slow or failing, without blocking the entire agent workflow?
Implement per-server timeouts so a single slow server doesn't stall the whole session, allow the agent/LLM to proceed with partial results and inform the user a specific capability is temporarily unavailable, and use circuit-breaker patterns to stop repeatedly retrying a consistently failing server within a session.
98. What is the significance of idempotency in MCP tool design, particularly for state-changing operations, given potential retries?
Since network failures/timeouts can cause a host to retry a tool call without certainty the original request succeeded, tools that create/modify state should be designed idempotently (e.g., accepting a client-generated idempotency key) to prevent duplicate side effects (like double-charging a payment or creating duplicate records) from retried calls.
99. How would you design a comprehensive testing/CI pipeline for an MCP server before it's published to a public registry?
Automated schema validation tests for every tool definition, unit tests for handler logic, integration tests using the MCP Inspector or a scripted test client simulating realistic multi-call sequences, security scanning of dependencies, and a manual review checklist covering description clarity, least-privilege permission scoping, and error message quality before publish.
100. Looking forward, what skills should a senior engineer develop to be well-positioned for MCP-based agentic system architecture roles?
Deep familiarity with the MCP spec and at least one SDK, strong API/system design fundamentals (since good tool design is fundamentally good interface design), practical experience with LLM tool-calling behavior and its failure modes, security engineering principles (least privilege, authZ, injection defense), and hands-on experience building and operating at least one production RAG and one production agentic system end to end.
Resources
LLM Fundamentals & Research
- "Attention Is All You Need" — the original Transformer paper
- Chinchilla scaling laws paper (Hoffmann et al.)
- Hugging Face Transformers documentation and course
- Andrej Karpathy's "Let's build GPT" and neural network video series
- DeepLearning.AI short courses on LLMs, RLHF, and fine-tuning
RAG
- RAGAS documentation (evaluation framework)
- LangChain and LlamaIndex documentation on RAG pipelines
- Anthropic's "Contextual Retrieval" engineering blog post
- MTEB (Massive Text Embedding Benchmark) leaderboard
MCP
- Official MCP specification: https://modelcontextprotocol.io
- MCP SDKs and MCP Inspector on GitHub (modelcontextprotocol org)
- Anthropic's MCP announcement and engineering blog posts
Practice & Mock Interviews
- Build a small end-to-end RAG project (ingest → chunk → embed → retrieve → generate → evaluate) from scratch
- Build a minimal MCP server (Python or TypeScript SDK) exposing 2–3 real tools and connect it to Claude Desktop
- Practice explaining trade-offs out loud — most senior interviews probe reasoning and trade-off awareness, not just definitions
About the Author
Himanshu Agarwal works at the intersection of applied AI engineering, RAG systems, and LLM-powered product development, and creates in-depth technical learning resources for engineers preparing for senior AI/ML and GenAI interviews.
Explore the Full Bundle
This guide covers the core 300 questions — the MCP, RAG & LLM Mastery Bundle goes deeper with full system design walkthroughs, annotated code projects, mock interview scripts, and downloadable cheat sheets for last-minute revision.
👉 Get the MCP, RAG & LLM Mastery Bundle
If this guide helped you, sharing it with someone else prepping for interviews is always appreciated.
Top comments (0)