DEV Community

Mads Hansen
Mads Hansen

Posted on

A successful ChatGPT database query can still be wrong

A plausible number is not a verified number.

A ChatGPT database query can run successfully and still answer the wrong business question.

Common causes:

  • the wrong metric definition
  • an inherited tenant or environment scope
  • stale upstream data
  • a month or timezone boundary
  • duplicate or late-arriving records
  • a truncated result
  • a confident summary of a provisional number

Before sharing the answer, reconcile it:

  1. write the metric, population, exclusions, time window, timezone, and currency
  2. bind it to an approved source and definition version
  3. record source freshness and upstream completion
  4. inspect typed results before the prose
  5. compare with an independent control
  6. explain material differences with a movement bridge
  7. preserve row count, truncation, filters, and trace evidence
  8. require a separate approval before publication

The best outcome is not always “the number matches.”

Sometimes it is: “The difference is explained, the source is late, and this answer is provisional.”

Full checklist: ChatGPT database query reconciliation

Top comments (0)