DEV Community

Ben Halpern
Ben Halpern Subscriber

Posted on

AWS S3 appears to be down

I'd add an image showing the chart from downdetectpr... But S3 is down, so I can't πŸ˜’

No status update yet from AWS

Top comments (11)

Collapse
 
farmer_sneed_b58f87019ef1 profile image
Farmer Sneed

S3 was the most consistent thing in my life. I don't know what to think anymore.

Collapse
 
melroy89 profile image
Melroy van den Berg

This is why I just host everything myself.

Collapse
 
n_eal_ee3cbb38cc8f5310c profile image
N E A L

Obviously

Collapse
 
airtonix profile image
Zenobius Jiricek

🀣🀣🀣🀣🀣🀣

Collapse
 
melroy89 profile image
Melroy van den Berg

Well it's true. I hate big tech

Collapse
 
rasvi24 profile image
rasvi24

The 11 9s are for data. Replicated a cross regions. Not for the availability of server to handle requests.

Collapse
 
daveparr profile image
Dave Parr

Finally hit the other side of that 5 nines reliability with the old 0.001% state huh? Definitely a post worthy scenario at those odds πŸ˜‰

Collapse
 
mauricebrg profile image
Maurice Borgmeier

12:46 PM PDT We are investigating Increased API Error Rates for Multiple services in the US-EAST-2 Region.
1:06 PM PDT Between 12:27 PM and 12:52 PM PDT we experienced increased API error rates for PUT and GET requests to S3 in the US-EAST-2 Region. Other AWS Services that use PUT and GET S3 APIs were also affected. The error rates have recovered, and we continue to investigate root cause.
1:19 PM PDT Between 12:27 PM and 12:52 PM PDT we experienced increased API error rates for PUT and GET requests to S3 in the US-EAST-2 Region. Other AWS Services that use PUT and GET S3 APIs were also affected. The error rates have recovered and all services are operating normally. Engineers were automatically engaged immediately and began mitigating the impact and investigating the root cause in parallel. We will post a final update as we validate root cause.
7:38 PM PDT We wanted to provide some additional information. Beginning at 12:27 PM, we experienced increased error rates for S3 GET/PUT requests made in the US-EAST-2 Region. This issue also impacted other AWS Services who use these APIs as part of service operations. The S3 issue was resolved at 12:52 PM, when S3 error rates returned to normal operations. Since then, we have been working to fully understand the root cause. This issue was caused by a latent software issue in a subsystem of S3 that is responsible for assessing metadata (such as versioning) during S3 PUT and GET operations. We have already implemented mitigations, which include removing the software issue in the S3 request path, and have identified new testing to detect these types of software issues in the future.

Source: health.aws.amazon.com/health/statu...

Seems like there was an issue in one AWS Region (us-east-2 - Ohio) that impacted reads and writes in that region. Fortunately not a global issue.

Collapse
 
fonsahpageo profile image
Fonsah Pageo

It's back guys
I just created a bucket, put an object, viewed, deleted. All good

Collapse
 
bntstr profile image
Bntstr

fml

Collapse
 
sanam2226 profile image
Ajay Pillai

Wasn’t it 11 - 9’s