DEV Community

Ben Halpern
Ben Halpern

Posted on

"There have been a couple of customers whose caches have just disappeared without us doing anything"

Software development is a shit show sometimes. The title of this article is part of a response I got when a cloud service we were using just wasn't there anymore. Here's the whole response, with some info redacted to protect the (surely frustrated themselves) innocent.

It looks as though your cache has been deleted (apparently on [redacted]). I'm guessing you didn't delete your cache. I need to investigate this properly, because we seem to have been getting some phantom cache deletion requests from [redacted] in the last few days -- there have been a couple of other customers whose caches have just disappeared without us doing anything or apparently them doing anything either. Which is obviously very worrying. I've added extra logging to try to track down what's going on, but I've not caught a case right as it happened yet.

In the meantime, the quickest way to get you going again is for you just to remove and re-add the [redacted] to your app, which will create a new cache. We'll sort out some reduction in charges for this month to cover the inconvenience, and I'll get back to you as soon as I've tracked down what's really going on.

In addition to the issues on their end, we sort of ignored the error at first, so we were serving our app without a critical cache for a little while. And then when I got into the investigation, I tried to make a quick change successfully crashed the app in production for about a minute, and that isn't easy to do with our CI setup.

The company's support was very fast with their response. I'm thankful for the wonderful support once I got in touch with them. Note to self: Do this sooner.

It has been an all-around "one of those weeks" at dev.to. But I'm really excited for Scott Hanselman's "Ask Me Anything" scheduled for today at 2pm EST. He's definitely one of my programming heroes. I'm hoping nothing goes wrong. 🙃

Top comments (0)