The drift detection feature is something I wish existed when I was building my test framework. I ended up with a different problem in the same space — not mocking AI responses, but validating that data stays consistent across layers (UI → API → DB). I used set theory on the DB state to mathematically prove that exactly one record changed per operation.
Your point about "the other six services are live and quietly making your test suite a lie" really resonates. In my case it was simpler — just three layers — but the same principle applies: if you're only mocking one thing, you're testing your mocks, not your system.
Curious about one thing: does AIMock support asserting on the sequence of calls across services? Like verifying that the LLM was called before the vector DB, not after?
as far as I'm aware there's no dedicated way to assert call sequence across services right now. the journal does log all calls across LLM, A2A, vector etc. so you could read it after the test and check ordering yourself but that's more of a workaround. pls create a feature request in the repo!
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
The drift detection feature is something I wish existed when I was building my test framework. I ended up with a different problem in the same space — not mocking AI responses, but validating that data stays consistent across layers (UI → API → DB). I used set theory on the DB state to mathematically prove that exactly one record changed per operation.
Your point about "the other six services are live and quietly making your test suite a lie" really resonates. In my case it was simpler — just three layers — but the same principle applies: if you're only mocking one thing, you're testing your mocks, not your system.
Curious about one thing: does AIMock support asserting on the sequence of calls across services? Like verifying that the LLM was called before the vector DB, not after?
as far as I'm aware there's no dedicated way to assert call sequence across services right now. the journal does log all calls across LLM, A2A, vector etc. so you could read it after the test and check ordering yourself but that's more of a workaround. pls create a feature request in the repo!