A comprehensive, up-to-date reference for QA engineers, SDETs, ML/AI test engineers, and hiring managers. Covers classical ML testing, LLM/GenAI evaluation, RAG systems, agentic AI, bias & safety testing, MLOps pipelines, and the current (2026) tooling landscape.
Table of Contents
- Foundations of AI/ML Testing
- Data Quality & Data Testing
- Model Testing & Validation
- Metrics & Evaluation
- LLM & Generative AI Testing
- Prompt Engineering & Prompt Testing
- RAG (Retrieval-Augmented Generation) Testing
- AI Agents & Agentic Workflow Testing
- Bias, Fairness, Safety & Responsible AI
- Adversarial, Security & Red-Teaming
- MLOps, CI/CD & Test Automation
- Performance, Scalability & Monitoring
- Tools & Frameworks (2026 Landscape)
- Behavioral & Scenario-Based Questions
- Further Resources
1. Foundations of AI/ML Testing
1. How is testing AI systems different from testing traditional software?
Traditional software has deterministic input-output mappings defined by explicit logic, so a fixed test oracle (expected output) usually exists. AI systems — especially ML models and LLMs — are probabilistic, learned from data, and can produce different (sometimes non-deterministic) outputs for the same input. Testing AI therefore shifts from "does output X match expected Y exactly" to statistical validation: accuracy thresholds, distributional checks, behavioral/invariant testing, and continuous monitoring for drift, since the "correct" behavior is learned rather than coded.
2. What are the main categories of AI testing?
Data testing (quality, bias, leakage), model testing (accuracy, robustness, fairness), integration testing (model + application), behavioral testing (invariance, directional expectation, minimum functionality), non-functional testing (latency, throughput, cost, security), and — for generative systems — output quality/safety evaluation (hallucination, toxicity, factuality).
3. What is a "test oracle problem" in AI testing, and how do you address it?
It's the difficulty of knowing the "correct" output for a given input when the ground truth is itself uncertain or expensive to obtain. Approaches include: metamorphic testing (checking relationships between outputs rather than exact values), using human-labeled gold sets, cross-referencing with a stronger reference model, statistical/threshold-based acceptance, and consensus/majority voting across multiple evaluators.
4. What is metamorphic testing and why is it useful for ML/AI?
Metamorphic testing defines relationships ("metamorphic relations") between multiple related inputs and their expected outputs instead of needing a single ground truth. For example, if you rotate an image slightly, a classifier's prediction should stay the same; if you increase a loan applicant's income in a credit model, the approval probability shouldn't decrease. It's especially useful when no oracle exists.
5. What is the difference between verification and validation in an AI/ML context?
Verification asks "did we build the model correctly?" (code correctness, pipeline correctness, reproducibility). Validation asks "did we build the correct model?" (does it solve the business problem, generalize to real-world data, and meet acceptance criteria).
6. What is model drift, and what types exist?
Model drift is the degradation of model performance over time as the real world diverges from training assumptions. Key types: data drift (input feature distributions change), concept drift (the relationship between inputs and target changes), label drift (distribution of the target variable changes), and upstream data drift (changes in data pipelines/schemas feeding the model).
7. What is the difference between training, validation, and test sets, and why does test-set contamination matter?
Training data fits model parameters; validation data tunes hyperparameters and drives model selection; the test set gives an unbiased final performance estimate and must never influence training decisions. Contamination (test data leaking into training, common with web-scraped LLM pretraining corpora) inflates reported performance and is a major 2026 concern for benchmark trustworthiness.
8. What is the "shift-left" principle as applied to AI testing?
Moving quality checks earlier in the lifecycle — validating data schemas and label quality before training, unit-testing feature engineering code, and running lightweight model sanity checks in the same PR pipeline as data/model changes — rather than only evaluating a finished model right before release.
9. Why can't traditional code coverage metrics be applied directly to ML models?
Code coverage measures which lines of code execute, but a model's "logic" lives in learned weights, not in branches of code. Instead, ML testing uses concepts like data coverage (are all relevant input regions/slices represented), behavioral coverage (are all expected behaviors tested), and neuron/activation coverage (used in some deep-learning test research) as rough analogues.
10. What is the CACE principle ("Changing Anything Changes Everything") in ML testing?
Because ML models are tightly coupled statistical systems, changing one input feature, one hyperparameter, or even the data order can shift the model's behavior in ways unrelated components don't anticipate — unlike modular software where a change is usually locally contained. This motivates full-pipeline regression testing after any change, not just unit tests of the changed component.
2. Data Quality & Data Testing
11. What are the key dimensions of data quality to test before training?
Completeness (missing values), accuracy (correctness vs. source of truth), consistency (no contradictory records), uniqueness (duplicates), timeliness (freshness), and validity (conforms to schema/format/range constraints).
12. How do you test for data leakage?
Check for overlap between train/test sets (exact or near-duplicate rows), features that are proxies for the target computed using future information (temporal leakage), and improper preprocessing (e.g., scaling/imputing using statistics from the full dataset before the train/test split). Automated leakage detectors compare feature importance spikes and unusually high train/test correlation.
13. How do you test for class imbalance and why does it matter?
Compute class distribution and compare against a threshold (e.g., minority class < 5%). Imbalance can cause models to optimize for the majority class while achieving poor recall on minority classes — critical when the minority class is the one that matters (fraud, disease detection). Testing includes verifying stratified sampling and confirming per-class metrics, not just aggregate accuracy.
14. What is a data schema/contract test, and why do modern data pipelines need them?
A schema test validates that incoming data matches an agreed contract — column names, types, ranges, nullability, categorical value sets. In 2026 data-mesh and streaming architectures, contract testing (e.g., via Great Expectations, Pandera, or dbt tests) at pipeline boundaries prevents silently broken upstream data from corrupting model training or inference.
15. How do you test for bias in training data?
Analyze representation across protected/sensitive attributes (age, gender, ethnicity, geography), check label quality for annotator bias, compute statistical parity across subgroups, and use tools like Fairlearn, AIF360, or What-If Tool to surface disparities before training even begins.
16. What is synthetic data testing, and what are its risks?
Synthetic data is generated (often by another model) to augment or replace real data for privacy or scarcity reasons. Testing must verify statistical fidelity (does it match real-data distributions), utility (does a model trained on it perform comparably on real test data), and privacy leakage (can real records be re-identified/memorized). A 2026 risk is "model collapse" from repeatedly training on synthetic/AI-generated data, degrading diversity over generations.
17. How would you test a data pipeline used for LLM pretraining or fine-tuning corpora?
Validate deduplication effectiveness, PII/toxic-content filtering, license/copyright compliance scanning, language identification accuracy, tokenization correctness, and benchmark-contamination checks (ensuring eval sets aren't present in training data).
18. What is feature store testing?
Testing that feature values computed offline (batch, for training) match those computed online (real-time, for inference) — the "training-serving skew" problem. This includes point-in-time correctness (no future leakage) and freshness SLAs for online features.
19. How do you test data versioning and lineage?
Verify that each model artifact is traceable to an exact dataset version/hash (using tools like DVC, LakeFS, or Delta Lake time travel), and that lineage metadata correctly reflects transformations applied, enabling reproducibility and audit/rollback.
20. What techniques exist to detect outliers and anomalies in input data before they reach a model?
Statistical methods (z-score, IQR), density-based methods (DBSCAN, Isolation Forest), autoencoder reconstruction error, and rule-based range/format validation — often run as a real-time guardrail in the inference pipeline, not just offline.
3. Model Testing & Validation
21. What is behavioral testing of ML models, and what are the CheckList categories?
Popularized by the CheckList framework (Ribeiro et al.), behavioral testing checks model capabilities directly rather than only aggregate accuracy. Three key test types: Minimum Functionality Tests (MFT) — simple, targeted cases; Invariance Tests (INV) — output should not change under label-preserving perturbations (e.g., typos, name swaps); Directional Expectation Tests (DIR) — output should change in a predictable direction (e.g., adding negation should flip sentiment).
22. What is A/B testing in the context of ML models, and what pitfalls should you watch for?
Randomly routing traffic between a challenger and champion model and comparing business/quality metrics in production. Pitfalls: insufficient sample size/statistical power, novelty effects, network/interference effects between arms, and metric selection that doesn't reflect true business impact (e.g., optimizing engagement while degrading long-term trust).
23. What is shadow testing (shadow deployment) and when would you use it?
Running a new model in parallel with the production model on live traffic without serving its outputs to users, comparing predictions offline. It's ideal for validating latency, stability, and prediction differences under real traffic before any user exposure, with zero user risk.
24. What is canary testing for ML models?
Gradually rolling out a new model to a small percentage of production traffic, monitoring key metrics closely, and automatically rolling back if thresholds are breached — before ramping to 100%.
25. How do you test for overfitting and underfitting?
Compare training vs. validation/test performance curves (a large gap indicates overfitting; poor performance on both indicates underfitting); use learning curves across training set sizes; apply cross-validation to check variance across folds; and test on genuinely out-of-distribution holdout data.
26. What is cross-validation, and what are common variants used in testing?
A resampling technique that partitions data into k folds, training on k-1 and validating on the remainder, rotating through all folds to get a robust performance estimate. Variants: stratified k-fold (preserves class balance), time-series/rolling-window CV (respects temporal order, critical for forecasting models), and group k-fold (prevents leakage when samples are correlated, e.g., same patient/user in multiple rows).
27. How do you test model robustness to distribution shift?
Evaluate on deliberately out-of-distribution or perturbed test sets (different time periods, geographies, demographics, sensor noise), measure performance degradation, and use techniques like domain adaptation testing or covariate shift detection (e.g., population stability index, KL divergence between training and production feature distributions).
28. What is adversarial robustness testing?
Testing whether small, often imperceptible input perturbations (adversarial examples) can flip a model's prediction. Techniques include FGSM (Fast Gradient Sign Method), PGD (Projected Gradient Descent), and black-box query-based attacks; robustness is measured via metrics like attack success rate and minimum perturbation distance.
29. How do you test explainability/interpretability of a model?
Validate that explanation methods (SHAP, LIME, Integrated Gradients, attention visualization) produce consistent, faithful explanations — e.g., check that features flagged as important actually change the prediction when perturbed (fidelity), and that explanations are stable for similar inputs (stability).
30. How do you regression-test a model after retraining?
Maintain a fixed "golden" evaluation set with known expected behaviors (including edge cases and previously fixed bugs), compare new-model metrics against the previous model's baseline with defined tolerance thresholds, and flag any behavioral regressions on critical slices even if aggregate accuracy improves.
31. What is slice-based testing / subgroup analysis, and why does aggregate accuracy hide problems?
Aggregate accuracy can mask poor performance on specific data slices (a minority demographic, a rare product category, low-light images). Slice-based testing evaluates metrics separately across meaningful subgroups to catch these hidden failures — tools like Fairlearn, TensorFlow Model Analysis, and Robustness Gym automate this.
32. What is calibration testing, and why does it matter for probabilistic models?
Calibration checks whether predicted probabilities match real-world frequencies (a model predicting 80% confidence should be correct ~80% of the time). Tested via reliability diagrams and Expected Calibration Error (ECE); poorly calibrated models are risky in decision-critical applications like medical diagnosis or credit scoring, even if accuracy looks fine.
33. How do you test a recommendation system differently from a classifier?
Beyond accuracy metrics (precision@k, recall@k, NDCG, MAP), recommendation testing must cover diversity, novelty, serendipity, popularity bias, cold-start behavior (new users/items), and feedback-loop effects where the model's own outputs shape future training data.
34. What is champion-challenger testing?
An ongoing production pattern where the current best model (champion) is continuously compared against one or more candidate models (challengers) on live or shadow traffic, with promotion criteria defined in advance to reduce subjective decision-making about when to ship a new model.
35. How do you validate a computer vision model beyond top-line accuracy?
Test per-class precision/recall, confusion matrices for commonly confused classes, robustness to image corruptions (blur, brightness, occlusion, rotation — e.g., via ImageNet-C style benchmarks), bounding-box IoU thresholds for detection tasks, and fairness across demographic attributes when applicable (e.g., face-related systems).
4. Metrics & Evaluation
36. What are precision, recall, F1, and when do you prioritize one over another?
Precision = TP/(TP+FP) — how many predicted positives were correct; Recall = TP/(TP+FN) — how many actual positives were found; F1 is their harmonic mean. Prioritize recall when missing a positive is costly (disease screening, fraud), precision when false alarms are costly (spam filters flagging legitimate email).
37. What is ROC-AUC vs. PR-AUC, and when should you prefer PR-AUC?
ROC-AUC plots true positive rate vs. false positive rate across thresholds; PR-AUC plots precision vs. recall. PR-AUC is preferred for highly imbalanced datasets because ROC-AUC can look deceptively good when negatives vastly outnumber positives.
38. What is a confusion matrix and how do you use it in testing?
A table cross-tabulating predicted vs. actual classes. It's used to spot systematic error patterns (e.g., a model consistently confusing two similar classes) that a single aggregate metric would hide.
39. What metrics matter for regression models?
MAE (mean absolute error, robust to outliers, interpretable in original units), RMSE (penalizes large errors more), MAPE (percentage-based, but unstable near zero), and R² (proportion of variance explained). Choice depends on whether large errors should be penalized disproportionately.
40. How do you test time-series forecasting models?
Use time-respecting (walk-forward/rolling-origin) validation rather than random splits, evaluate with MAPE/SMAPE/MASE, test against naive baselines (e.g., "predict yesterday's value"), and check residuals for autocorrelation indicating unmodeled structure.
41. What is the difference between offline and online evaluation metrics?
Offline metrics (accuracy, F1, AUC) are computed on static historical data before deployment. Online metrics (click-through rate, conversion, latency, user retention) are measured on live traffic and reflect real business impact, which can diverge from offline metrics due to feedback loops and distribution shift.
42. What is statistical significance testing in model comparison, and why is it necessary?
Comparing two models' metrics on a single test set can be misleading due to sampling noise. Techniques like paired t-tests, bootstrap resampling, or McNemar's test establish whether an observed performance difference is statistically significant rather than random variation.
5. LLM & Generative AI Testing
43. How does testing an LLM-powered application differ from testing a classical ML classifier?
LLM outputs are open-ended natural language (or code, images, audio) rather than a fixed label set, so exact-match assertions rarely work. Testing shifts to: semantic similarity to reference answers, LLM-as-judge scoring, rubric-based grading, human evaluation, and behavioral checks (does the output follow instructions, avoid hallucination, stay in policy) — layered with the same non-functional concerns (latency, cost per token, safety).
44. What is "LLM-as-a-judge," and what are its limitations?
Using a (typically stronger) LLM to score or compare candidate outputs against criteria or a reference answer, at a scale human review can't match. Limitations: position/verbosity bias (judges favor longer or first-listed answers), self-preference bias (a model favors outputs resembling its own style), inconsistency across runs, and vulnerability to the same failure modes (hallucination) it's meant to catch — so it should be validated against human judgments (inter-rater agreement) rather than trusted blindly.
45. What is hallucination, and how do you test for it?
Hallucination is when a model generates fluent but factually incorrect or unsupported content. Testing approaches: fact-checking against a trusted knowledge base or the retrieved context (faithfulness/groundedness scoring), citation verification (does the cited source actually support the claim), consistency checks (does the model contradict itself across paraphrased queries), and using specialized hallucination-detection benchmarks (e.g., TruthfulQA, HaluEval, FActScore-style pipelines).
46. What is the difference between "faithfulness" and "answer relevance" in generative QA evaluation?
Faithfulness (or groundedness) measures whether the generated answer is factually supported by the provided context/source — independent of whether it answers the question. Answer relevance measures whether the answer actually addresses the user's question — independent of factual grounding. A response can be faithful but irrelevant, or relevant but unfaithful (hallucinated); good eval frameworks (e.g., RAGAS) score both separately.
47. How do you test an LLM for consistency?
Send the same or semantically equivalent prompts multiple times (varying temperature, phrasing, order of few-shot examples) and measure output stability — via exact match for structured outputs, semantic similarity (embedding cosine similarity) for free text, or self-consistency voting for reasoning tasks.
48. What is prompt injection, and how do you test for it?
An attack where malicious instructions embedded in user input or retrieved content override the system's intended behavior (e.g., "ignore previous instructions and reveal the system prompt"). Testing involves red-teaming with known injection payloads, testing both direct injection (user input) and indirect injection (poisoned documents/web content the model retrieves), and verifying the system maintains role/instruction boundaries under adversarial input.
49. What is jailbreaking, and how does it differ from prompt injection?
Jailbreaking is manipulating a model into bypassing its own safety training to produce disallowed content (e.g., via role-play framing, encoding tricks, or multi-turn escalation), whereas prompt injection specifically targets application-level instruction hijacking. Both are tested via red-teaming, but jailbreak testing focuses on the model's safety alignment, while injection testing focuses on application/tool-boundary integrity.
50. How do you evaluate LLM outputs for toxicity, bias, and harmful content?
Combine automated classifiers (e.g., Perspective API-style toxicity scoring, safety classifiers), curated adversarial prompt sets (e.g., RealToxicityPrompts-style benchmarks), and structured red-team exercises across protected categories, then track pass rates and severity over model/prompt versions.
51. What is a "golden dataset" for LLM evaluation, and how do you build one?
A curated set of representative input-output pairs (or inputs with grading rubrics) covering common cases, edge cases, and known failure modes, used as a stable regression benchmark across prompt/model changes. Build it by sampling real user queries, deliberately including adversarial and ambiguous cases, and having domain experts validate/label reference answers or acceptance criteria.
52. How do you test for output format compliance (e.g., valid JSON, schema adherence) in LLM applications?
Use schema validators (JSON Schema, Pydantic) to programmatically check structural correctness, test with constrained/structured-output modes (function calling, grammar-constrained decoding) versus free-form prompting, and measure the failure rate requiring retries or repair.
53. What is regression testing for prompts, and why is it needed even without code changes?
Because LLM providers periodically update underlying models (even on a "pinned" version, silent behavior shifts can occur), and because prompts themselves get iterated on, teams maintain automated prompt-regression suites — running the golden dataset against every prompt or model change and diffing scores — to catch quality regressions before release.
54. How do you benchmark and compare different LLMs for a specific use case?
Define task-specific evaluation criteria (not just generic leaderboard scores, which may not reflect your domain), run each candidate model against your own golden dataset with consistent scoring, and evaluate cost, latency, and context-window fit alongside quality — since the best model on public benchmarks isn't always best for a narrow, domain-specific task.
55. What is the difference between intrinsic and extrinsic evaluation of LLMs?
Intrinsic evaluation measures a model's output quality against reference metrics directly (perplexity, BLEU/ROUGE, judge scores). Extrinsic evaluation measures downstream task/business impact (did the AI assistant increase resolution rate, did the summarizer reduce review time) — extrinsic metrics are ultimately what matter but are more expensive and slower to collect.
56. What are BLEU, ROUGE, and BERTScore, and what are their limitations for evaluating modern LLM output?
BLEU and ROUGE are n-gram overlap metrics originally built for translation/summarization; BERTScore uses contextual embeddings for semantic similarity. All three correlate weakly with human judgment for open-ended generation, penalize valid paraphrases, and don't capture factual correctness or instruction-following — which is why LLM-as-judge and task-specific rubrics have largely supplanted them for chat/agent evaluation in 2026, though they're still used for translation and constrained summarization tasks.
57. How do you test multi-turn conversational AI systems?
Beyond single-turn quality, test context retention across turns, graceful handling of topic switches, recovery from user corrections, memory consistency (not contradicting earlier statements), and degradation as conversation length approaches context-window limits — often via scripted multi-turn conversation scenarios and simulated user agents.
58. What is "context rot" or long-context degradation, and how do you test for it?
The phenomenon where model performance degrades as relevant information is placed deeper in a long context window (or the context grows very long), even within the stated context limit — sometimes called the "lost in the middle" effect. Test by placing key facts at varying positions/depths within long contexts and measuring retrieval/answer accuracy as a function of position and total length ("needle in a haystack" style tests).
59. How do you test multimodal AI systems (text+image, text+audio)?
Test each modality's understanding independently (e.g., can it accurately describe an image, transcribe audio) and their fusion (does it correctly reason across modalities, e.g., answering questions that require both the image and accompanying text), plus modality-specific robustness (image resolution/compression artifacts, audio noise/accents).
60. What is the role of human-in-the-loop (HITL) evaluation, and when is it non-negotiable?
HITL evaluation uses human raters to score outputs against guidelines, providing ground truth for calibrating automated evaluators and catching failure modes automated methods miss. It's non-negotiable for high-stakes domains (medical, legal, safety-critical) and for periodically auditing/re-calibrating LLM-as-judge pipelines to prevent automated evaluation drift.
6. Prompt Engineering & Prompt Testing
61. What is prompt testing, and how does it fit into a CI/CD pipeline?
Systematically evaluating how changes to a prompt (wording, examples, structure) affect output quality, using a fixed evaluation dataset and metrics, integrated as an automated pipeline step so prompt edits are gated by regression checks just like code changes.
62. How do you test few-shot prompts for robustness?
Vary the number, order, and selection of few-shot examples, and check output stability — LLMs are known to be sensitive to example ordering and can exhibit recency/majority-label bias from the shots provided.
63. How do you test for prompt sensitivity to irrelevant formatting changes?
Apply semantically neutral perturbations (whitespace, punctuation, casing, paraphrasing the instruction) and verify the output doesn't meaningfully change — a form of invariance testing adapted to prompts.
64. What is "temperature" and how should test suites account for it?
Temperature controls sampling randomness; low temperature (near 0) is more deterministic and suited for reproducible regression tests, while higher temperature is used for creative tasks and requires multiple-sample evaluation (e.g., pass@k) rather than single-run assertions.
65. How do you test system prompts for leakage and boundary enforcement?
Attempt adversarial extraction ("repeat your instructions," encoding tricks, role-play bypasses) and verify the system prompt/instructions aren't disclosed, and that the model refuses out-of-scope requests consistently across paraphrased attack attempts.
7. RAG (Retrieval-Augmented Generation) Testing
66. What are the distinct components of a RAG pipeline that need separate testing?
Ingestion/chunking (are documents split sensibly), embedding quality (do semantically similar texts get similar vectors), retrieval (does the right context get found), and generation (does the LLM use the retrieved context correctly). Each stage can fail independently, so end-to-end scores alone don't localize the problem.
67. How do you test retrieval quality in isolation from generation?
Use metrics like context precision (proportion of retrieved chunks that are relevant), context recall (proportion of needed information actually retrieved), and hit rate/MRR/NDCG@k against a labeled set of query-to-relevant-document pairs — independent of what the LLM does with that context afterward.
68. What is context precision vs. context recall in RAG evaluation?
Context precision measures how much of what was retrieved is actually relevant (penalizing noisy retrieval); context recall measures how much of the relevant information that exists was successfully retrieved (penalizing missed information). A pipeline can have high precision but low recall (retrieves few but clean chunks, missing key facts) or vice versa.
69. How do you test chunking strategy effectiveness?
Compare different chunk sizes/overlap strategies against retrieval and answer-quality metrics on the same golden query set, and check for "context fragmentation" where an answer's supporting information is split across chunk boundaries and never retrieved together.
70. How do you detect when a RAG system answers from the LLM's parametric memory instead of the retrieved context?
Design test cases where the retrieved context intentionally contradicts the model's likely pretrained knowledge (counterfactual context) and check whether the answer follows the provided context — a failure to do so indicates over-reliance on parametric memory, a groundedness failure.
71. What is the RAGAS framework, and what does it measure?
RAGAS is a popular evaluation framework for RAG pipelines that computes reference-free metrics including faithfulness, answer relevance, context precision, and context recall, typically using an LLM judge, enabling automated regression testing of retrieval and generation quality together or separately.
72. How do you test a RAG system's behavior when no relevant document exists?
Include queries in the test set with intentionally no correct answer in the knowledge base, and verify the system abstains or says it doesn't know rather than hallucinating a plausible-sounding but unsupported answer.
8. AI Agents & Agentic Workflow Testing
73. What additional challenges do AI agents (that use tools, plan multi-step tasks, and act autonomously) introduce for testing?
Agents chain multiple LLM calls and external tool invocations, so errors compound across steps; testing must cover tool-selection correctness, argument correctness, error recovery/retry behavior, task decomposition quality, and whether the agent knows when to stop or ask for clarification — not just final-output quality.
74. How do you test tool/function-calling correctness in an agent?
Verify the agent selects the correct tool for a given intent, passes correctly formatted and semantically valid arguments, handles tool errors/timeouts gracefully, and doesn't call tools with side effects (payments, deletions) inappropriately — including deliberately injecting tool failures to test recovery paths.
75. What is trajectory evaluation for agents, and how does it differ from outcome evaluation?
Outcome evaluation only checks whether the agent achieved the correct final result; trajectory evaluation examines the full sequence of intermediate steps/decisions/tool calls, since an agent can reach a correct answer via an inefficient, unsafe, or accidentally-correct path that would fail under slightly different conditions.
76. How do you test for infinite loops or runaway behavior in autonomous agents?
Enforce and test step/time/cost budgets, verify the agent detects repeated failed attempts and escalates or stops rather than retrying indefinitely, and simulate ambiguous or unsolvable tasks to confirm graceful termination.
77. How would you test a multi-agent system where agents communicate with each other?
Test individual agent behavior in isolation (unit level), pairwise communication protocols/message formats (integration level), and full-system emergent behavior under realistic and adversarial scenarios (system level) — watching for miscommunication, deadlocks, and error amplification across agent handoffs.
78. How do you test agent memory (short-term and long-term)?
Verify short-term (in-context) memory correctly carries relevant state across a session, long-term (persisted) memory correctly stores and retrieves prior interactions, and that outdated/incorrect memories can be corrected or expired rather than causing persistent errors.
9. Bias, Fairness, Safety & Responsible AI
79. What are the common fairness metrics used to test ML models, and why might they conflict?
Demographic parity (equal positive-prediction rates across groups), equalized odds (equal true/false positive rates across groups), and predictive parity (equal precision across groups) are common metrics. They can mathematically conflict with one another (proven impossibility results) except in special cases, so teams must choose the fairness definition most appropriate to their context and be explicit about the trade-off.
80. How do you test for proxy discrimination?
Check whether seemingly neutral features (zip code, name, school) correlate strongly with protected attributes and drive disparate outcomes even when protected attributes are excluded from the model directly; techniques include correlation analysis and counterfactual fairness testing (does the prediction change if only the protected attribute is altered, holding a causal model of the rest constant).
81. What is counterfactual fairness testing?
Testing whether a model's output changes when a protected attribute (or a proxy) is swapped in an otherwise identical input (e.g., changing an applicant's gender or a customer's name in a resume-screening tool) — a stable, unbiased model should give consistent outputs across such counterfactuals.
82. What frameworks/regulations should AI testers be aware of in 2026?
The EU AI Act (risk-tiered obligations for AI systems, phased implementation through 2026-2027 with high-risk system requirements including testing, documentation, and post-market monitoring), NIST AI Risk Management Framework, ISO/IEC 42001 (AI management systems), and sector-specific rules (e.g., FDA guidance for AI/ML-based medical devices). Testers should verify their evaluation and documentation practices satisfy the applicable regime for their system's risk tier.
83. How do you build a responsible-AI test checklist for a new model before release?
Cover: data provenance/consent, bias/fairness across relevant subgroups, robustness/adversarial testing, explainability documentation, privacy (PII leakage, memorization testing), safety/harm testing (toxicity, misuse potential), human oversight mechanisms, and a documented model card summarizing intended use, limitations, and evaluation results.
84. What is membership inference and model inversion testing, and why does it matter for privacy?
Membership inference tests whether an attacker can determine if a specific record was in the training set (a privacy leak); model inversion tests whether an attacker can reconstruct sensitive training data from model outputs/gradients. Both are tested by simulating attacker access levels and measuring attack success rate — critical for models trained on sensitive personal data.
85. How do you test for training-data memorization in LLMs?
Probe the model with prefixes of known training documents (or canary strings deliberately inserted during training) and measure verbatim-completion rate; extraction attack benchmarks quantify how much and how easily memorized content can be regurgitated, which matters for both privacy and copyright risk.
10. Adversarial, Security & Red-Teaming
86. What is AI red-teaming, and how does it differ from traditional penetration testing?
Red-teaming systematically probes an AI system for harmful, unsafe, or policy-violating outputs and behaviors — combining traditional security concerns (data exfiltration, unauthorized access) with AI-specific attack surfaces (jailbreaks, prompt injection, bias elicitation, hallucination triggering). Unlike traditional pen-testing's binary "exploited/not exploited," AI red-teaming often deals with graded, subjective harm severity and requires domain/policy expertise alongside security skills.
87. What are common categories in an AI red-team taxonomy?
Harmful content generation, privacy violations (PII leakage/memorization), misinformation/factual manipulation, bias/discrimination elicitation, jailbreaks/safety bypass, prompt/system injection, excessive agency (unauthorized tool use), and denial-of-service via resource-exhausting prompts.
88. What is data poisoning, and how do you test defenses against it?
An attack where an adversary injects malicious samples into training data (or a RAG knowledge base) to manipulate model behavior. Testing/defense: anomaly detection on training data, provenance tracking, influence-function analysis to identify high-impact suspicious samples, and testing model behavior against known poisoning patterns in a controlled sandbox.
89. What is model extraction/stealing, and how would you test a deployed model's exposure to it?
An attacker queries a model API extensively to train a substitute model that mimics its behavior, potentially stealing IP or enabling further attacks. Test exposure by measuring how quickly a surrogate model trained on your own API's outputs approaches original performance, and validate rate-limiting/watermarking/output-perturbation defenses.
90. How do you incorporate automated red-teaming tools into a CI pipeline?
Maintain a library of adversarial prompt templates and attack generators (e.g., automated jailbreak-mutation tools), run them against every model/prompt version as a gating test, track pass/fail rates and severity trends over time, and route new discovered failures back into the golden regression dataset.
11. MLOps, CI/CD & Test Automation
91. What does a mature CI/CD pipeline for ML/AI systems typically test at each stage?
Pre-commit/PR: unit tests for feature engineering/preprocessing code, data schema validation. Build: reproducibility checks, dependency/environment validation. Training: data quality gates, training convergence sanity checks. Post-training: full evaluation suite against golden/holdout sets, fairness/bias checks, behavioral tests. Pre-deployment: shadow/canary testing, load testing. Post-deployment: continuous monitoring for drift, performance degradation, and automated alerting/rollback.
92. What is continuous evaluation (as opposed to one-time model evaluation)?
An ongoing pipeline that re-evaluates a deployed model (or LLM application) against fresh production samples and the golden dataset on a schedule or trigger (new model version, data drift detected, prompt change), rather than treating evaluation as a single pre-launch gate — necessary because both real-world data and, for LLM APIs, the underlying model itself can silently change over time.
93. How do you version and reproduce ML experiments for testing purposes?
Track code (git), data (DVC/LakeFS/data hashes), model artifacts and hyperparameters (MLflow, Weights & Biases), and environment (containerization) together so any evaluated result can be reproduced exactly — essential for debugging test failures and for audit/compliance.
94. What test automation strategy would you use for an LLM-powered feature shipping weekly?
A layered pyramid: fast, cheap deterministic checks (schema/format validation, latency budgets) on every commit; a mid-sized golden-dataset regression suite with LLM-as-judge scoring on every PR/merge; a broader nightly/weekly run including red-team probes, human-reviewed spot checks, and full fairness/bias audits before major releases — balancing feedback speed against evaluation depth.
95. How do you handle flakiness/non-determinism when writing automated tests for generative AI outputs?
Pin temperature/seed where supported for deterministic-mode tests, use similarity thresholds or LLM-judge rubrics instead of exact match, run multiple samples and assert on statistical properties (e.g., pass rate ≥ 90% across n runs) rather than single-run pass/fail, and separate "must always pass" safety-critical assertions from "quality trending" softer metrics.
12. Performance, Scalability & Monitoring
96. What non-functional aspects are unique to testing AI/LLM systems in production?
Token-based cost per request (and cost regression testing when prompts grow), tail latency under variable generation length, GPU/accelerator utilization and batching efficiency, rate-limit and quota handling for third-party model APIs, and graceful degradation (fallback to smaller/cheaper models) under load or provider outages.
97. How do you load-test an AI inference service, and what's different from typical API load testing?
Beyond standard throughput/latency/error-rate testing, account for highly variable response times (short vs. long generations), GPU memory/batch-size constraints that create nonlinear scaling behavior, and cost-per-request as a first-class metric alongside latency — plus testing autoscaling behavior specifically for GPU-backed infrastructure, which scales slower than typical stateless CPU services.
98. What should be monitored continuously for a production ML/LLM system, and how does monitoring feed back into testing?
Input data/prompt distribution drift, output distribution/quality drift (sampled and scored continuously), latency/cost/error-rate SLAs, safety-classifier trigger rates, user feedback signals (thumbs up/down, escalations, corrections), and model/provider version changes. Anomalies detected in monitoring should automatically generate new test cases added to the regression suite — closing the loop between production observability and pre-release testing.
99. How do you decide when a model or AI feature is "good enough" to ship?
Define acceptance criteria up front, tied to business impact, not just model metrics: minimum performance thresholds on the golden/holdout set, fairness parity within an agreed tolerance across subgroups, red-team pass rates on safety-critical categories, acceptable latency/cost envelopes, and a rollback/monitoring plan for post-launch issues — evaluated together as a release checklist rather than a single accuracy number, and ideally validated against a shadow or canary deployment before full rollout.
15. Further Resources
Foundational papers & frameworks
- Ribeiro et al., "Beyond Accuracy: Behavioral Testing of NLP Models with CheckList" (ACL 2020) — origin of MFT/INV/DIR behavioral testing
- Google, "Machine Learning Test Score: A Rubric for ML Production Readiness" — the CACE principle and production ML testing checklist
- "RAGAS: Automated Evaluation of Retrieval Augmented Generation" — faithfulness/context precision/recall metrics
- NIST AI Risk Management Framework (AI RMF 1.0) — nist.gov/itl/ai-risk-management-framework
- EU AI Act official text and implementation timeline — artificialintelligenceact.eu
Benchmarks & datasets
- TruthfulQA, HaluEval — hallucination benchmarks
- HELM (Holistic Evaluation of Language Models), Stanford CRFM
- MMLU, GPQA, BIG-Bench Hard — general LLM capability benchmarks (useful as a starting point, not a substitute for domain-specific eval)
- RealToxicityPrompts — toxicity evaluation
- Great Expectations "Expectation Gallery" — data quality test patterns
Tooling (open-source & commercial, 2026 landscape)
- Data quality: Great Expectations, Pandera, Soda Core, Monte Carlo, dbt tests
- Classical ML testing/fairness: Evidently AI, Deepchecks, Fairlearn, AIF360 (IBM), What-If Tool, Alibi Detect (drift)
- LLM/GenAI evaluation: RAGAS, DeepEval, promptfoo, LangSmith (LangChain), Braintrust, Arize Phoenix, TruLens, Confident AI, Weights & Biases Weave
- Red-teaming/security: Garak (NVIDIA), PyRIT (Microsoft), Giskard, Rebuff (prompt injection detection)
- Agent evaluation: LangSmith, AgentBench, τ-bench (tau-bench) for tool-use/trajectory evaluation
- Experiment/model tracking & MLOps: MLflow, Weights & Biases, DVC, Kubeflow, Seldon Core, BentoML
- Load/perf testing for inference: Locust, k6, vLLM benchmarking suite, LLMPerf
Where to keep up to date
- Anthropic, OpenAI, Google DeepMind model/system cards (published with each major model release) — these document each provider's own evaluation methodology and are a strong reference for how frontier labs structure safety/capability testing.
- Anthropic's documentation on building reliable, evaluated AI applications: docs.claude.com
- OWASP Top 10 for LLM Applications — owasp.org (updated regularly; the closest thing to an industry-standard LLM security checklist)
- ISO/IEC 42001:2023 (AI Management System standard) for organizations formalizing AI governance and audit processes
This guide reflects testing practices and tooling current as of mid-2026. The GenAI/agentic AI tooling space moves quickly — always verify tool feature sets and benchmark leaderboards directly before an interview or a real evaluation project, since specifics (best-in-class tools, current SOTA benchmark scores, regulatory deadlines) can shift within months.
Top comments (0)