Over four days in September our warehouse picked, packed and dispatched six hundred and twelve orders that customers had cancelled. About fourteen thousand pounds of goods, a large number of apologies, and a conversation with another team that began with them asking, reasonably, what do you mean you are reading it.
Orders live in a system owned by a different division. It has no API and there was no appetite to build one, so in 2022 somebody sensible arranged a read only login against a replica, and our dispatch service has selected from four of its tables every two minutes ever since. It works. It is fast, it costs nothing, and for four years it was the least troublesome integration we had.
In their September release they introduced soft deletes. A cancelled order is no longer removed from the table, it is marked with a deletion timestamp, which is a good change that makes their own auditing possible. Our query asks for orders with no dispatch state and takes what it gets, because until that release a cancelled order was an absent row. That was never a rule anyone wrote. It was a property of their implementation that we had turned into our contract without telling them or, honestly, without noticing.
Their release notes described the change accurately. They went to their consumers, and we were not on that list, because nothing on their side recorded that a replica had a reader. The credential was named after a project that had been renamed twice.
We now read a view, four columns wide, which they own and which their pipeline has a test for, and the raw tables are no longer granted to that login at all. An event feed is on their roadmap and the view will do until it arrives. We also went looking for the rest: three more integrations where we read another system's storage directly, one of them a file on a shared drive that a finance job writes at midnight. Each has a named owner on both sides now and an entry in the other team's release checklist, which is the only part of this that actually prevents a repeat.
Selecting from somebody else's database makes their internal design into your interface, with no version, no notice period and no counterparty. They can change it correctly, announce it properly, and still have no idea you exist.
– Sergey Shinder
Top comments (0)