DEV Community

Mads Hansen
Mads Hansen

Posted on

One hard question is a better database acceptance test than ten easy ones

Do not approve a PostgreSQL-to-ChatGPT connection because ten easy questions returned plausible answers.

Use one representative business question with:

  • a known answer
  • an explicit reporting cutoff
  • a tenant boundary
  • a join
  • a versioned metric definition
  • a meaningful stale, incomplete, or empty state

Then require evidence for identity, PostgreSQL role, source/replica, freshness, coverage, normalized filters, query limits, truncation, retries, and provenance.

Run the negative twin too: unauthorized tenant, missing partition, stale replica, unknown enum, and forced timeout. The final prose must preserve refusal, incomplete, stale, and unknown states instead of collapsing them into zero rows.

One hard question tests the contract. Ten easy questions mostly test fluency.

Full guide: Connect PostgreSQL to ChatGPT: the first-query acceptance test

Top comments (0)