Data-scientist who loves to use #datascienceforgood, especially in ecology, energy and the environment. Bonsai, gardening, bikes and music when I'm not at a keyboard.
12:46 PM PDT We are investigating Increased API Error Rates for Multiple services in the US-EAST-2 Region. 1:06 PM PDT Between 12:27 PM and 12:52 PM PDT we experienced increased API error rates for PUT and GET requests to S3 in the US-EAST-2 Region. Other AWS Services that use PUT and GET S3 APIs were also affected. The error rates have recovered, and we continue to investigate root cause. 1:19 PM PDT Between 12:27 PM and 12:52 PM PDT we experienced increased API error rates for PUT and GET requests to S3 in the US-EAST-2 Region. Other AWS Services that use PUT and GET S3 APIs were also affected. The error rates have recovered and all services are operating normally. Engineers were automatically engaged immediately and began mitigating the impact and investigating the root cause in parallel. We will post a final update as we validate root cause. 7:38 PM PDT We wanted to provide some additional information. Beginning at 12:27 PM, we experienced increased error rates for S3 GET/PUT requests made in the US-EAST-2 Region. This issue also impacted other AWS Services who use these APIs as part of service operations. The S3 issue was resolved at 12:52 PM, when S3 error rates returned to normal operations. Since then, we have been working to fully understand the root cause. This issue was caused by a latent software issue in a subsystem of S3 that is responsible for assessing metadata (such as versioning) during S3 PUT and GET operations. We have already implemented mitigations, which include removing the software issue in the S3 request path, and have identified new testing to detect these types of software issues in the future.
S3 was the most consistent thing in my life. I don't know what to think anymore.
This is why I just host everything myself.
🤣🤣🤣🤣🤣🤣
Well it's true. I hate big tech
Obviously
Finally hit the other side of that 5 nines reliability with the old 0.001% state huh? Definitely a post worthy scenario at those odds 😉
The 11 9s are for data. Replicated a cross regions. Not for the availability of server to handle requests.
Source: health.aws.amazon.com/health/statu...
Seems like there was an issue in one AWS Region (us-east-2 - Ohio) that impacted reads and writes in that region. Fortunately not a global issue.
It's back guys
I just created a bucket, put an object, viewed, deleted. All good
Wasn’t it 11 - 9’s
fml