DEV Community

Mads Hansen
Mads Hansen

Posted on

A new database connector can return a valid answer and still break the contract

A database connector migration can pass every setup check and still change the answer.

The new path authenticates, discovers the schema, and returns valid JSON.

But it may use a different default time range, tenant mapping, row limit, redaction rule, replica, or retry policy.

That is why connector migration needs shadow traffic, not only a cutover checklist.

A practical sequence:

  1. inventory the current tool, policy, and result contracts
  2. freeze representative requests and expected invariants
  3. replay negative paths offline
  4. mirror authorized reads to the candidate
  5. keep the existing connector responsible for the visible answer
  6. compare scope, metric, source, limits, redaction, freshness, and provenance
  7. classify every mismatch as expected, timing, contract drift, policy drift, or unknown
  8. promote an internal canary before wider routing
  9. test rollback before retiring the old path

Do not shadow writes or external actions unless the candidate has a real dry-run boundary. Duplicate side effects are not test coverage.

And do not compare only serialized output. Two answers can use different rows and both be correct because live data changed. What matters is whether the same business and security invariants held.

Full migration guide: ChatGPT connector alternatives: shadow traffic before cutover

Top comments (0)