“Show me twenty examples” sounds harmless.
The generated SQL adds LIMIT 20 without a stable order.
The database returns whichever rows happen to arrive first. The assistant finds a pattern, and the team treats that pattern as evidence.
But a row limit is not a sample design.
Physical layout, indexes, query plans, parallel workers, recent inserts, and cache state can all change which rows appear.
Before examples support a conclusion, define:
- the population and cutoff
- authorization scope
- the sampling method
- stable row identity
- seed and algorithm version
- strata and weights
- redaction
- what the sample cannot prove
Use deterministic ordering for debugging. Use a stable hash sample for repeatable pseudo-random selection. Use stratification when important cohorts must be represented. Oversample rare cases deliberately—and disclose the bias.
Most importantly, separate observation from inference.
“7 of 20 sampled cases had a missing category” does not automatically mean “35% of all cases are missing a category.” Use an approved aggregate when prevalence matters.
A sampling receipt should make the population, method, seed, version, limits, freshness, source coverage, and result checksum reviewable.
LIMIT protects the system. A sampling contract protects the conclusion.
Full guide: ChatGPT database queries need a deterministic sampling contract
Top comments (0)