Halloween is a good time for scary stories. The thing is, I don't have any ghost stories to tell. I have something much scarier: AWS bills 👻.
I skipped the stories about access keys pushed to GitHub and crypto miners spun up in every region. That's a topic for the security team. What scares me more are the cases where nobody stole anything. The account was secure, the code worked, and the bill still went through the roof.
Because AWS bills you for the work done, not for its value to your business. If something does the same work a million times, you pay a million times more.
I collected ten such stories, including my own. Together they add up to about $258,000 before refunds (and one of them is an especially sneaky one). Where the source gives the volume, I recalculated the amount from the AWS price list. Sometimes it matches to the cent. For every story I also checked whether the monster is still alive, because AWS has closed some of these traps since. All prices are for us-east-1. In European regions they are sometimes a bit higher.
At the end you'll find a 15-minute checklist. You can run it on your own account tonight. 🎃
Strangers knocking at the door
The first three stories have one thing in common: the bill was generated by someone the account owner didn't know and didn't invite.
1. The haunted house: an empty bucket for $1,300
Let me start with a classic. With a Polish twist, too.
In April 2024, Maciej Pocwierz created a private, empty S3 bucket in eu-west-1. He needed it for a simple proof of concept. Two days later he checked his billing page to make sure he was still within the Free Tier. He saw more than $1,300.
His bucket name was identical to a default value in the configuration of a popular open-source backup tool. Every installation that nobody had reconfigured tried to upload its backups to his bucket. Almost 100 million PUT requests in one day. Every single one rejected with 403 Access Denied.
From a permissions point of view, everything worked: no stranger wrote a single file. From a billing point of view, not so much, because the bucket owner paid for every rejected request.
AWS first replied that this was "expected behavior". In the end the bill was cancelled, as an exception. The story went viral, and Jeff Barr publicly admitted that it had to be fixed.
I recalculated: 100 M PUT × $0.005 per 1,000 requests = $500 per day. So $1,300 is about two and a half days at that rate. On top of that, requests without a region first went to us-east-1 and were redirected, which added cost. The order of magnitude matches.
Is the monster alive? No. It's the only monster in this list that AWS really killed. The change was announced on 13 May 2024, and by August 2024 it covered all S3 APIs and regions. Watch out for one detail, though: only 403 errors that come from outside your account or AWS Organization are free. If your own application hammers a bucket without permissions, you still pay.
Exorcism: unique bucket names (with a random suffix), not backup, data or test.
2. The mummy: a file that woke up after years
In July 2020, Chris Short opened the email with his bill. He usually paid about $23 a month. For June: $2,657.68.
A 13.7 GB virtual machine image sat in a public S3 bucket. Cloudflare stood in front of the bucket, so Chris assumed the CDN would cache everything. But Cloudflare's Free, Pro and Business plans don't cache files larger than 512 MB. So every download went straight to S3, and S3 charged for data transfer to the internet. More than 30.6 TB in two days: $1,011.59 on 23 June and $1,639.07 on 24 June.
Who was downloading that file? Nobody knows to this day. Corey Quinn summed it up: Chris didn't do anything wrong. AWS refunded everything except about $40.
I recalculated: the first 10 TB × $0.09/GB + the next 20.6 TB × $0.085/GB ≈ $2,650. Less than 1% off the bill.
The same thing, 2026 edition: in March, an r/aws user described how, after his site was promoted on a subreddit, someone downloaded files from his bucket non-stop for three days. About 160 TB and about $15,500. AWS first reduced the bill only partially, and the author still couldn't pay the rest. AWS waived the remaining amount only after six weeks, and after the author, on a commenter's advice, wrote to Jeff Barr. Transferring 160 TB alone costs about $12,000 according to the price list, so the order of magnitude matches. A caveat: this is an account from an anonymous Reddit user, with no breakdown of the rest of the amount.
Is the monster alive? Yes. The 2024 change doesn't help here, because downloading a file from a bucket someone can access is a valid, billable request.
Exorcism: don't expose S3 directly to the internet. Serve files through CloudFront with Origin Access Control, and check whether your CDN really caches large objects.
3. A swarm of bats: 207 million requests in three hours
Web Dev Cody, a YouTuber well known in the JavaScript world, woke up in March 2024 to an email from AWS Budgets. He had set an alert at about $10. The bill was $269, almost $260 of it CloudFront.
For roughly three hours, while Cody slept, his Icon Generator app received more than 207 million requests. The /community page alone got about 25 million of them, and those responses came from cache. The app didn't have to compute anything, yet every HTTPS request was billed. Cody suspected a DoS attack or a script. The traffic stopped on its own. The bill stayed, and Cody doesn't mention any refund.
I recalculated: the bill in his video shows a line of 207,146,239 HTTPS requests in the US region. × $0.01 per 10,000 requests = $207.15. That's exactly the amount on that line. To the cent. Plus $31.37 for data transfer (these are two lines from the part of the bill shown, not the full breakdown).
The alert worked as a notification, not as a brake. Threshold $10, bill $269, and the traffic stopped before Cody woke up. Thanks to the alert, though, he found out the same day. He added a rate-limit rule in AWS WAF and closed the case.
Is the monster alive? Weakened. Since 18 November 2025 CloudFront has flat-rate plans (from the free Free plan to Premium at $1,000 a month) with no overage charges. Blocked DDoS attacks and requests rejected by WAF don't count against the allowance. But this is an option, not the default. A plan covers a specific distribution, not your whole backend, and an existing distribution doesn't get one automatically. On classic pay-as-you-go, the monster is alive and well.
Exorcism: a rate-limit rule in AWS WAF, or a flat-rate plan for projects that don't make money from traffic.
Loops that feed themselves
4. The poltergeist: a function that calls itself
June 2022. Ruslan Gainutdinov ran a Next.js site with Lambda@Edge. By mistake, the function started calling its own CloudFront distribution. According to the author, the recursive requests ran at about 17,000 per second, each with a 30-second timeout.
The bill: about $4,500.
The most interesting detail is another one, though. Ruslan had a budget alert set at $300. The alert arrived when the charges were already $1,484. CloudFront charges can show up on your bill with a delay of up to 24 hours. The author calculated that if the alert had arrived on time, he would have paid about $400.
I recalculated: assuming 128 MB of memory, one 30-second invocation is 3.75 GB-s × $0.00005001 ≈ $0.00019. So $4,500 is about 24 million invocations, which is just under half an hour at a steady 17,000 per second. The rate was probably a peak, but the order of magnitude matches.
Is the monster alive? In this form, yes. AWS detects and stops Lambda loops (since July 2023 for SQS and SNS, since October 2024 for S3, and according to the current documentation also for custom EventBridge buses). A loop through CloudFront and HTTP is not on that list.
Exorcism: a rate limit in WAF on the distribution, a header or counter that breaks the recursion, and a CloudWatch alarm on invocation count (Lambda@Edge metrics live in the regions where the replicas ran). The invocation count grows immediately. The dollars show up hours later.
5. My own poltergeist
I know this story best, because I wrote it myself.
When I was learning serverless and event-driven architecture, I built a simple flow. A file lands in S3, the event triggers a Lambda, the Lambda starts a Step Functions state machine, and the state machine processes the file and writes the result back to S3.
Nothing suspicious. Except that I put the input prefix and the output prefix in the same place.
So writing the result started the next round of processing. I built an infinite loop and simply didn't notice it for a while. The wake-up call came by email: I had used up the Step Functions Free Tier. After a few dozen minutes of playing around. 😅
Luckily, I stopped my tests at that point. And it was me who stopped the loop, not AWS. On accounts with the classic Free Tier, running out of it doesn't stop anything, it just starts charging.
Scale: the Step Functions (Standard) Free Tier is 4,000 state transitions per month. A state machine with a few states uses that up in a few hundred executions, and a loop does those in minutes. After that, every 1,000 transitions cost $0.025.
I don't have a six-figure bill to show here. What I do have is a very personal memory of the moment I understood that automation repeats my mistake just as efficiently as it does useful work.
Is the monster alive? Yes. Lambda loop detection covers S3, but my chain went through Step Functions. I wouldn't assume detection catches it.
Exorcism: input and output in separate buckets, or at least in separate prefixes with a filter on the S3 event.
Good intentions, terrible bill
In this category nobody did anything stupid. Everyone was trying to do things properly.
6. The vampire: a NAT Gateway that drinks every gigabyte
November 2025. The Geocodio team built a new process that synced large datasets from EC2 to S3. They knew that data transfer between EC2 and S3 in the same region is free. And that's true.
A few days later, an alert came from AWS Cost Anomaly Detection. In one day, 20,167.32 GB went through a NAT Gateway.
So where did the bill come from? From the path the data took.
The instances were in a private subnet. If they had been in a public one, the traffic wouldn't have gone through NAT at all. A private subnet has no direct route out of the VPC. Its route table sends all traffic leaving the VPC (0.0.0.0/0) to the NAT Gateway. And S3 isn't part of your VPC. It's a service with a public address. Without an S3 Gateway Endpoint, every request to S3 goes through the NAT Gateway, then through the Internet Gateway, and only then reaches the public S3 endpoint.
An important detail: this traffic doesn't go out to the public internet. According to the Amazon VPC FAQ, traffic between AWS resources stays on the AWS network, even when it uses public addresses. And the transfer from EC2 to S3 itself is still free, exactly as the price list says.
You pay for something else. A NAT Gateway charges $0.045 for every gigabyte it processes, no matter where that gigabyte is going. The transfer was free. The gate it passed through was not.
An instance in a public subnet (with a public IP and a route to the Internet Gateway) wouldn't have this problem, because the Internet Gateway doesn't charge for data processing. And a Gateway Endpoint adds a direct route to S3 to the route table, bypassing NAT. It costs nothing.
I recalculated: 20,167.32 GB × $0.045 = $907.53. Exactly the amount Geocodio reports for that one day. In January 2026 they added that AWS refunded the full amount.
Is the monster alive? Yes. The Gateway Endpoint for S3 and DynamoDB is free, but you still have to add it yourself.
Exorcism: a Gateway Endpoint for S3 and DynamoDB in every VPC, ideally in your IaC template, so nobody forgets it.
7. The ghost in the logs: one print for $1,500
On 4 August 2020, Michael Malis, CEO of Freshpaint, got a budget alert email. It was the fourth day of the month, and the company had already spent more than $1,600 against a budget of $1,000 for the whole month.
Almost $1,500 went to CloudWatch. The cause: a print statement added during debugging that logged every event passing through the backend. It made it to production and generated terabytes of logs. The whole cost was a single line item: DataProcessing-Bytes, meaning log ingestion into CloudWatch at $0.50 per GB ($0.63 in Frankfurt).
I recalculated: $1,500 / $0.50 = 3,000 GB, about 3 TB of logs. The author writes about "multiple terabytes". It matches.
An important detail: shortening retention wouldn't have changed anything. You pay for ingesting logs, not only for storing them.
Is the monster alive? Yes. The Infrequent Access log class is cheaper, but volume still costs money.
Exorcism: log level set in configuration, not in code, and an alarm on log volume for every log group.
8. The curse of defaults: CloudTrail for $200,000
In my opinion, this is the scariest story in the list. Precisely because it's so boring.
Bharath Nadar described how, while onboarding a security tool, someone accepted its default configuration: CloudTrail Data Events for all reads and writes in all S3 buckets, across the whole AWS organization. Full audit, compliance, security. Who would object?
The thing is, CloudTrail charges $0.10 for every 100,000 data events. And buckets that applications read billions of times a month generate billions of events.
The cost grew slowly: from about $1,000 a month in 2024 to about $30,000 a month in 2025. There was no sudden spike, so nothing raised an alarm. The author put it this way: "It didn't spike. It crept." He found the problem manually, in Cost Explorer, at the end of 2025. Total loss: about $200,000. The fix took 5 minutes: switching logging to writes only.
I recalculated: the author reports 8.98 billion S3 events in December, in us-east-1 alone. × $0.10 per 100,000 = $8,980 in one month and one region. That's consistent with the roughly $10,000 a month the author gives for a single production account.
A caveat: this is one person's account, with no company name. The mechanism and the numbers are consistent, though.
Is the monster alive? Yes. Data Events for high-volume S3 reads will always be expensive.
Exorcism: narrow Data Events with selectors to specific buckets and to writes. And before you enable a new tool, check how much the services it turns on will cost.
Monsters you can't see
9. The zombie: you deleted it, but it keeps walking
October 2024. Elliott King was building a side project on AWS Amplify Gen 2 and got a bill of about $1,125.
He deleted the sandbox environment with npx ampx sandbox delete. The DynamoDB tables disappeared. The Amazon OpenSearch Service domains stayed and kept running. Every new sandbox added another domain.
AWS adjusted the bill as a one-time courtesy. A long discussion grew under his post on Hacker News, so Elliott was definitely not alone.
Is the monster alive? I don't know. At the end of October 2024 the author wrote that the problem still existed. I found no confirmation that it has been fixed. What I do know is that the zombie is always alive when you trust a delete command without checking what it left behind.
Exorcism: after every environment deletion, check resources by tags (Tag Editor in Resource Groups). And set a budget on every account, including the one "just for fun".
10. The invisible monster: $30,000 for Claude Opus
This is the freshest story. May 2026, The Register. One of its readers used Claude Opus through Amazon Bedrock. He had Cost Anomaly Detection configured. The first sign that something was wrong was the invoice: $30,141.33 for April, plus about $675 in other charges.
Why did anomaly detection stay silent? Third-party models in Bedrock are billed through AWS Marketplace. And at that time Cost Anomaly Detection didn't see Marketplace charges at all. An AWS spokesperson confirmed this directly to The Register.
On top of that, AWS Activate credits quietly covered the first $8,026.54. When they ran out, nobody got a notification.
Is the monster alive? Weakened. On 19 August 2026 AWS announced that Cost Anomaly Detection now covers third-party models in Bedrock, with no extra setup. The documentation still says, however, that other Marketplace charges are not monitored. For those you need AWS Budgets.
Exorcism: a budget on the total account cost (it includes Marketplace) and a separate alert for credits running out.
Why the alarm didn't ring
When I put these stories side by side, I saw something more interesting than the amounts. Some people had no alarm at all. But where there was an alarm, it failed in four ways.
1. The alarm was late. Ruslan had a $300 alert that arrived at $1,484. Cody had a $10 alert and paid $269. That wasn't a bug. According to the documentation, AWS Budgets updates its data up to three times a day, usually every 8 to 12 hours. AWS itself writes that you can exceed the threshold before you get the email. Cost Anomaly Detection uses Cost Explorer data and needs up to 24 hours to detect an anomaly.
2. The alarm was blind. The Register reader had Cost Anomaly Detection, but it didn't see Marketplace costs. Credits hid the first $8,000. The alarm was looking the other way.
3. The monster crept. CloudTrail grew for more than a year. Anomaly detection learns your spending pattern and, according to the documentation, accounts for natural growth. So a steady, slow increase is easy to treat as normal. (The author didn't say whether he had anomaly detection. But even if he had, I wouldn't count on it.)
4. The monster was new. Few people know this detail: for a newly used service, Cost Anomaly Detection needs 10 days of history before it starts detecting anomalies. A service you turned on yesterday that got out of control today is invisible to it.
The alerts that did work (Cody, Geocodio, Freshpaint) gave one thing: knowledge within hours or days, not at the end of the month. None of them stopped anything. An alert is a smoke detector, not a fire extinguisher.
So did AWS finally give us a brake?
Partly. For years the answer was "there is no hard spending limit". Budget Actions can apply an IAM policy or an SCP, or stop specific EC2 and RDS instances. They won't stop Lambda, S3 or Bedrock.
On 16 September 2026, AWS introduced a new, project-based way of creating accounts. On the paid plan you can set a spend limit for a project. The minimum is $20 or AWS's estimate of your spend, whichever is higher. The limit applies to pre-tax costs and doesn't include credits. When you reach it, AWS pauses the project and stops its resources. There are also optional earlier thresholds, triggered a few days before the forecast overrun: blocking new resources, pausing idle resources, and even pausing the biggest cost drivers, including Lambda and Bedrock.
Sounds great. But there are three catches:
- Spend limits work only in the new account model (AWS Settings). For traditional accounts, which in practice means every existing company, the comparison in the AWS documentation says it plainly: not available.
- A spend limit is a type of AWS Budgets, so it uses the same delayed data. A loop that burns thousands of dollars in half an hour will manage to do so.
- If a project is paused and you do nothing for 90 days, AWS permanently deletes its data. A brake that, instead of slowing you down, scraps the car.
For learning and experiments, it's a great direction. In production, you still have to rely on yourself.
Are these monsters still alive?
| Monster | Mechanism | Status in October 2026 |
|---|---|---|
| 1. Empty bucket | Charges for rejected 403 requests | ⚰️ Dead for requests from outside your account and organization (since 2024) |
| 2. Mummy (13.7 GB) | Data transfer from S3 to the internet | 🧟 Alive |
| 3. 207 M requests | A charge for every CloudFront request | 🩹 Weakened: flat-rate plans since 11.2025, if you choose them |
| 4. Lambda@Edge | Recursion over HTTP | 🧟 Alive: loop detection doesn't cover this path |
| 5. My loop | S3, Lambda and Step Functions | 🧟 Alive |
| 6. Geocodio | NAT Gateway instead of a Gateway Endpoint | 🧟 Alive |
| 7. Freshpaint | Log ingestion into CloudWatch | 🧟 Alive |
| 8. CloudTrail | Data Events for S3 reads | 🧟 Alive |
| 9. OpenSearch zombie | Resources that survived delete
|
❓ No evidence of a fix |
| 10. Claude on Bedrock | Charges outside anomaly detection | 🩹 Weakened: detection since 08.2026 |
One dead, two weakened, six alive, and one I know nothing certain about. AWS does fix these traps, but slowly. And in two cases (1 and 10) only after someone described the problem publicly.
Monster hunting in 15 minutes
Every one of these monsters leaves a trace in your cost data. Open Cost Explorer, select the last 3 months, set Group by: Usage type and look for these traces. Usage types have a region prefix, for example EUC1-DataProcessing-Bytes for Frankfurt.
| Monster | What to look for | What to do |
|---|---|---|
| 🧛 Vampire (NAT) | A usage type with NatGateway-Bytes in its name (under EC2-Other) |
Add a Gateway Endpoint for S3 and DynamoDB |
| 🏺 Mummy (transfer) |
DataTransfer-Out-Bytes for S3 |
Move public files behind CloudFront |
| 👻 Ghost in the logs |
DataProcessing-Bytes for CloudWatch |
Find the log group with the highest volume |
| 🧙 CloudTrail curse | CloudTrail cost growing month over month | Check your Data Events selectors |
| 🦇 Bats (CloudFront) | CloudFront request cost higher than transfer cost | WAF rate limit or a flat-rate plan |
Then a few quick checks in the terminal. Which VPCs have an S3 endpoint and of what type (compare the result with the list from aws ec2 describe-vpcs):
aws ec2 describe-vpc-endpoints --region eu-central-1 --filters Name=service-name,Values=com.amazonaws.eu-central-1.s3 --query 'VpcEndpoints[].{Id:VpcEndpointId,Vpc:VpcId,Type:VpcEndpointType}'
What exactly your CloudTrail trail logs (look for DataResources or AdvancedEventSelectors with S3):
aws cloudtrail get-event-selectors --trail-name YOUR_TRAIL_NAME
Whether you have a budget, an anomaly monitor and a subscription that will send an alert:
aws budgets describe-budgets --account-id YOUR_ACCOUNT_ID
aws ce get-anomaly-monitors
aws ce get-anomaly-subscriptions
An anomaly monitor without a subscription won't notify anyone. If you're missing a budget, a monitor or a subscription, you know what you're doing tonight. 😉
What this all means
After ten stories, one thing is clear: AWS almost always billed exactly as its price list says. In the Geocodio and Cody stories, my recalculation matches to the cent. Monsters rarely come from billing errors. More often they come from defaults and from a price list read without the context of your own architecture.
That's why before every deployment I want to be able to answer three questions:
- What in this architecture can run over and over without delivering any new value?
- How will I know it's happening?
- How will I stop it, and who makes that call?
And after deployment I do three things:
- I set up AWS Budgets and Cost Anomaly Detection. Knowing that they are smoke detectors, not fire extinguishers. They ring late and they don't see everything.
- I look at the mechanism, not the service. An amount next to "S3" tells you little. What matters is whether you pay for storage, requests or transfer. An amount next to "EC2-Other" tells you even less.
- I review trends, not just spikes. The most expensive story in this list didn't have a single spike. It grew calmly for a year.
One more thing. In at least four of these stories, AWS gave the money back. But usually as a one-time courtesy, often only after a public post. That is not a FinOps strategy.
I do this kind of review every day in CloudPouch, because this exact pain is why I built the app. But whatever tool you use, do the review. Before your story ends up in the next list like this one.
And if you have your own AWS bill horror story (no leaked keys!), share it in the comments. I'd love to read it. 🎃
Sources
- Maciej Pocwierz, How an empty S3 bucket can make your AWS bill explode, InfoQ coverage and The Register coverage
- AWS, announcement of 13 May 2024, announcement of August 2024 and Billing for Amazon S3 error responses
- Chris Short, The AWS bill heard around the world
- r/aws, $15,000 S3 Bill for DDoS, AWS reduced my $15k S3 bill to $10.5k after a DDoS, Update: My $15.5k AWS S3 DDoS bill has been fully resolved
- Web Dev Cody, I got my first DDoS (and what you can do to help prevent it) (the bill is visible around 0:40)
- AWS, CloudFront flat-rate pricing plans, plan documentation and CloudFront pricing
- Ruslan Gainutdinov, I DDoSed myself using CloudFront and Lambda@Edge and got a $4.5k bill and the Hacker News discussion with the author's comments
- AWS, Lambda recursive loop detection and the S3 announcement of October 2024
- Mathias Hansen (Geocodio), The $1,000 AWS mistake and the Amazon VPC FAQ (traffic to public AWS addresses stays on the AWS network)
- Michael Malis (Freshpaint), Avoiding a Surprise AWS Bill
- Bharath Nadar, A $200,000 AWS bill nobody noticed
- Elliott King, Amplify overcharge
- The Register, Bedrock and a hard place: Claude adventure leaves AWS user staring down $30K invoice
- AWS, Cost Anomaly Detection supports third-party models on Amazon Bedrock and the Cost Anomaly Detection documentation
- AWS, AWS Budgets: data refresh frequency and Budget Actions
- AWS, the new getting-started experience, 16 September 2026, spend limits in AWS Settings and comparison of sign-up options
- AWS pricing: S3, VPC and NAT Gateway, CloudWatch, CloudTrail, Lambda, Step Functions
Originally published in Polish on Serverless Polska.
Top comments (1)
The debug print one is the scariest to me because nothing looks broken. The site works, the logs are just very enthusiastic.
Do you put a cost alarm on log ingestion specifically, or only on the total bill?