For help understanding this article or how you can implement auth
and similar security architectures in your services, feel free to
reach out to me via the community server.
AWS just released a supposed fix for S3 bucket squatting by utilizing what they are calling Account Regional Namespaces. I don't understand the hype, and now I'm going to explain why.
Broken: S3 Bucket Names are Global
S3 bucket names are global. Not global to your account. Not global to your region. Global to the entire AWS partition — every account, every region, every customer who has ever existed on AWS.
This was not a deliberate design philosophy. It was a default from 2006 that nobody corrected. S3 launched when AWS was essentially a startup with Amazon as its main customer. Global uniqueness was the path of least resistance. Nobody asked whether it would cause problems at scale, because at the time "scale" meant a hundreds or thousand developers, not millions of accounts and decades of production workloads.
But, that default is still in place today.
AWS's relationship with the S3 naming model, circa every year since 2008.
The sad truth is, nobody needs global bucket names. There is no use case that requires your bucket name to be universally unique across every AWS customer on the planet. The value of global uniqueness flows entirely in one direction: it must have simplified the original implementation. The cost of global uniqueness flows in the other direction: two decades of pain for every customer who has ever tried to name a bucket something sensible.
The abomination lives on because someone probably said "Wouldn't be cool if you could expose your S3 bucket publicly?" And for that the bucket name would have to be in the URL, and therefore globally unique (and also require that the bucket name be lowercase and RFC 7553 compliant). This is true but also irrelevant. S3 doesn't even support TLS for custom domains. So there is no way to actually serve an asset such as https://assets.mycompany.com directly from your S3 Bucet. None, full stop. Let's break that down, there are three parts to that URL — HTTPS, your domain, and something that maps to the S3 bucket. It has always been, and still is only PICK 2.
Anyone who needs a public URL with a real domain and HTTPS already is using CloudFront as a reverse proxy. As a matter of fact, every SPA out there, must be using CloudFront in order to achieve HTTPS or they must not be using a custom domain. The only suitible URL is the CloudFront distribution's alias, not the S3 bucket name. The bucket name is internal plumbing that nothing outside your AWS account should ever reference directly. I'm here to tell you that not only are global bucket names a mistake, there is actually an easy way to fix it. One has to wonder why AWS hasn't.
The people who think they need global bucket names are the people using S3 Virtual Hosting — mybucketname.s3.amazonaws.com — which does have TLS, but on AWS's domain, not theirs. And of course, there the sad case for supporting this pattern indefinitely because AWS is much nicer than some other cloud providers that constantly deprecate actually required features, such as DNS Zone hosting. Although in recent times that hasn't held up as much, and gives credence to AWS dropping the concept as it would have direct Security and Reliability wins. Not to mention straight out improvement by reducing complexity. There is no case for making it the architectural foundation of an object storage service used by billions of production workloads. And as we will see shortly, exposing that endpoint directly comes with its own expensive problem that CloudFront eliminates entirely.
The reality is none of the following are tradeoffs you agreed to. They are the consequences of a default, set in 2006, that nobody changed. The cost has landed on you ever since. And boils down to basically one core concept.
Name squatting
The boring version: the bucket name you want — mycompany-prod-logs, myapp-assets, opentofu-state — was registered years ago by someone who no longer works at the company that registered it. AWS has no mechanism for name reclamation. That name is gone until the current owner deletes the bucket, which may never happen. So what you might think, just choose a new name, like you would choose a new username, or website domain. This isn't a new problem after all.
But the reality is: bucket names are predictable, and predictable names are claimable before you need them, and it turns out some bucket names you actually very much need.
The researchers at Aqua Security demonstrated this at Black Hat USA 2024, calling it Bucket Monopoly. AWS services, themselves, create S3 buckets automatically use naming patterns derived from your account ID. Account IDs are not secret — they appear in IAM role ARNs, error messages, S3 URLs, and CloudTrail logs. And while good hygiene means keeping your AWS account ID obscured, the bucket names themselves must be completely public. S3 Virtual Hosting resolves every bucket as a DNS subdomain (mybucket.s3.amazonaws.com), Certificate transparency, and passive DNS collectors observe and index those queries continuously. And while they might not have caught everything, any bucket that has ever received traffic via Virtual Hosting has a name that likely exists in a DNS database outside your control.
Many naming patterns were vulnerable:
- Athena:
aws-athena-query-results-{account-id}-{region}— data query results - Elastic Beanstalk:
elasticbeanstalk-{region}-{account-id}— application build artifacts - AWS Config:
config-bucket-{account-id}— compliance and configuration records - CloudFormation, Glue, EMR, SageMaker, ServiceCatalog, and CodeStar all also have had similar patterns
The complete impact ranged from data exfiltration to remote code execution to full-service takeover. AWS has patched many of these services after disclosure.
The CDK case may be the worst case. AWS's own infrastructure-as-code tool hack wrapper (because actually the CDK isn't the IaC tool) bootstraps a staging bucket with a name that was never random:
cdk-hnb659fds-assets-{account-id}-{region}
The qualifier hnb659fds is a hardcoded constant in CDK's bootstrap template. It has never changed. Anyone who knows your account ID knows your CDK staging bucket name. If that bucket does not exist — because you deleted it, or becouse you have not bootstrapped yet, or because someone cleaned up an old environment — an attacker can claim it. CDK will then use that bucket to store and retrieve CloudFormation templates. The attacker injects a malicious template. CDK deploys it using an IAM role with broad permissions. Full account takeover.
Aqua Security found over 38,000 accounts susceptible. The vulnerability was present for years before being fixed in CDK v2.149.0 in July 2024.
To be clear, an attacker who learns your AWS Account ID, can register those bucket names before you deploy the service. AWS will see that the bucket exists, trust it, and the route your data into the attacker's bucket. This is happening even without your knowledge. Have you actually checked that every bucket AWS is secretly sending data to is self-owned by your account? Probably not, you probably don't even know which buckets AWS is using.
Security Through Obscurity
I thought it would go without saying, but I'm sure someone will bring it up: "Keep your bucket name obscure" is not a defense, since you can figure out these buckets by just using AWS services. And worse, the bucket name shows up in website hosting CNAMEs, presigned urls, and other places. It is publicly available.
And of course the inverse is also a problem. S3 bucket names carry implicit trust. When your infrastructure reads configuration from my-config-bucket, it assumes the content is authoritative because the name is correct. The global namespace means that assumption is structurally unsound — the name and the owner are not bound to each other in any durable way. An attacker who controls a bucket your infrastructure reads from doesn't need to exfiltrate anything. They inject. Your service pulls the configuration, trusts it, and acts on it.
This is not abstract. Consider the pattern of storing IAM permission mappings in S3 and distributing them via OU StackSets across an AWS organization. Something I actually just wrote about doing. An attacker who controls that bucket — whether by squatting the name, claiming it after a deletion, or exploiting a misconfigured access policy — can inject a permissions map that adds their own identity as a trusted principal. The StackSet propagates the poisoned configuration to every account in the org. Their CICD pipeline assumes the role via OIDC federation. Full organization-wide access, delivered through the normal configuration path, with no credentials created and no anomalous API calls.
This is the same pattern that made Clownstrike's botched configuration update in 2024 so severe. A trusted delivery mechanism pushed configuration that every endpoint pulled and acted on without independent verification. The delivery channel was correct. The content was not. Millions of machines followed instructions from a source they had no reason to distrust.
The difference is that Clownstrike's delivery infrastructure was their own, and the configuration was negligent, not malicious. Whereas the S3 version of this attack does not require compromising the infrastructure owner at all, it only requires claiming a bucket name.
The global namespace is what makes this entire attack class possible. In a correctly scoped namespace, your bucket names are yours, and an attacker in a different account cannot claim them. AWS built a shared global pool and then built their own services on top of it using predictable names, inheriting the vulnerability they created.
Security misconfiguration
The public access model exists because bucket names are global. Since any AWS account can reference your bucket by name, making a bucket readable without credentials makes it readable by everyone — which is occasionally intentional and routinely catastrophic.
The deeper problem: S3's access control system has never cleanly separated "accessible by my AWS account" from "accessible by the public internet." That distinction is not a first-class concept in S3. It has to be constructed from a combination of overlapping controls, each added at a different point in S3's history, each with its own interaction rules:
-
Bucket policies — grant access to specific principals or to
*(everyone) -
ACLs — a separate, older system with its own grantees, including the confusingly named
AuthenticatedUsersproperty - Block Public Access — four separate boolean flags that apply restrictions over policies and ACLs, added only in 2018 as a retroactive guardrail
- Object Ownership — controls whether ACLs are enforced at all, added later still
- IAM Policies — scopes permissions to principals with IAM authority.
Each layer was added to contain the blast radius of the previous one. None of them establish "private to my account" as the starting point. They establish "open to everything" as the starting point and ask you to correctly configure the restrictions. Miss one flag, misread one grantee, inherit one policy from a module you didn't write — and the bucket is likely public.
I like this article from 6 years ago talking a bit about that
IAM access summarized
But then you realize, this is just how IAM works, it isn't how S3 works at all. Sure whether or not IAM grants access is part of the picture, but where's the rest of it? I was trying to find a document in the AWS Docs that does a good job of explaining. There isn't one. There are over One Hundred Pages on access control in S3 alone. Don't believe me, count them. To be fair we have more than one page on similar Authorization concepts in the Authress KB. However, arguably what we designed has to be significantly more complex, since it has to handle literally every possible authorization scenario.
This is not a configuration problem. It is an architecture problem. It is a security problem. The controls are layered on top of a model that was never designed to be private.
And while the likelihood of getting it wrong has gone down significantly, the trade-off has been increased burden on configuration and setup.
Historical Hacks
Each problem identified by the community attracted a from AWS patch. But no one said they were the right patch.
Forced random suffixes
For buckets operated by AWS Services, you have no recourse, but for buckets you manage for your own platform, you have a small, but not very satisfying alternative. Because the global pool is full of names claimed by other accounts, you cannot have the names you want. my-app-assets is taken. opentofu-state is taken. prod-logs is taken. The community's answer to the problem, years before AWS even started to take any approach, is to use the only reliable strategy available — append a random suffix and stop trying to name things sensibly: my-app-assets-8f2a3c, opentofu-state-a1b2c3, prod-logs-9e4d71.
A list of your S3 buckets is now a list of opaque identifiers. Understanding which bucket belongs to which service requires either tagging discipline — which degrades over time — or reading OpenTofu state, which is stored in an S3 bucket with a random suffix. Not to mention this only gets around the creation problem, and doesn't remotely address the security angle.
This is not a novel problem. Discord ran the same experiment with usernames. Their original system appended a four-digit discriminator to every display name: warren#0088. Globally unique, unambiguous, machine-friendly. I don't remember anyone that could actually remember their discriminator. I can't imagine how many friend requests failed because users entered the wrong tag. With only 10,000 discriminators available per name, popular names of course ran out.
Discord's fix was not to make the discriminator longer. They separated the unique identifier — the username, used for backend lookups — from the display name, which is human-readable and non-unique. The part that needed global uniqueness was the lookup mechanism. The part humans see and share does not need to be globally unique at all.
S3 never made this distinction. The bucket name is simultaneously the unique global identifier, the human-readable label, and the public URL component. When all three concerns are collapsed into one string that must be globally unique across every AWS customer, you get my-app-assets-8f2a3c. That is your discriminator.
Forced predictable suffixes
For us we've taken a slightly different approach. And that's because random suffixes cannot be dynamically used at read time, are not idempotent, and that means usually hard-coding this string in multiple places. Or worse, I've seen many implementations attempt to export the generated S3 name from the infrastructure process to somewhere else, effectively coupling disparate systems that had no business being coupled together.
Our approach is to add the AWS Account ID, the Region, and an internal consistent identifier to ever bucket we create. Now everyone will understand what that means. For example, you can imagine you choose something like -${accountId}-${region}-un1que1d. Is that clever? Not really, but it is far better than having every bucket have a random ID.
The ExpectedBucketOwner property
One hack AWS added was integrating a new parameter into the S3 bucket APIs, which could validate ownership on bucket related actions such as Creation, PutObject, and GetObject. Released in Oct 2020, every S3 API call could now include the expected AWS account ID of the bucket owner. If the bucket exists but belongs to a different account, the call fails. You add this header to your SDK calls, your bucket policies, your presigned URL logic. The problem of AWS created buckets was so bad, that AWS needed an internal security fix for the problem. And this helped a little bit for us users as well. It isn't a real solution though, just something hacked on top.
The problem with this hack though, is that it is security you have to opt into, and if you are using some library or reusable module, good luck assuming that made it in.
CDK v2.149.0
In the July 2024 fix for the CDK boostrap, AWS merged a change that adds a condition to the CDK bootstrap role, preventing the attacker-controlled-bucket scenario. However, the fix still required teams to re-run cdk bootstrap. Any environment bootstrapped with CDK v2.148.1 or earlier and not yet re-bootstrapped remains vulnerable. The hack qualifier still remains hnb659fds, but you can change it, if you want to.
Block Public Access
By 2018, the pattern was clear: teams were misconfiguring bucket policies and ACLs what seemed like on-purpose, as if they were on a mission to win an award. Objects were going public, breaches were making headlines, and the individual controls were too granular and too easy to get wrong. AWS's response was to add a meta-level override: Block Public Access — four boolean flags that sit above all bucket policies and ACLs and veto any access grant that would expose objects to the public internet. To be clear, these flags don't affect the bucket at all, the affect the ability for you to change those other insecure properties on the bucket.
BlockPublicAcls, IgnorePublicAcls, BlockPublicPolicy, RestrictPublicBuckets. Each flag a different angle on the same problem.
It is a kill switch. It works, for the most part. It was necessary because the model it was bolted onto had no safe default — the access system started too easy to open and required teams to correctly configure the restrictions, which teams reliably failed to do at scale. Block Public Access does not change that model. It adds a blunt override and calls it a fix. AWS enabled it by default for new accounts in 2022.
Paying for unauthorized access
Did you know until 2024, if someone attempted to access your AWS S3 bucket, even if it was never public, would still incur a charge for you? This massive oversight was fixed under the radar, and you can read more about it in the release Amazon S3 will no longer charge for several HTTP error codes. How that ever got off the ground in the first place is honestly shocking.
OU: Block Public Access
Finally, only last year, did AWS release the ability for AWS Organizations to turn off the incredibly insecure configuration by utilizing one of the S3 Org level policies. Now you can actually be sure you don't accidentally get it wrong, or I guess also find out if you did much sooner than you would have.
The entire history of S3 naming advice, summarized.
Are these hacks? Yes, yes they are. That is because the default considerations for using S3 require more configuration then lesser used strategies. If you want your bucket to be public, you configure less than you do if you want it to stay private. If you want to make sure you are secure and writing to your own bucket, you need to add properties, rather than remove.
What AWS Just Shipped
The biggest challenges with all of these hacks is — that with each new one being introduced, it required every service, product, application, and library to directly integrate that change. That's because every API, architecture decision, and code path had to account for this change. These weren't just hacks AWS made to solve the problem, these were bad hacks that pushed the burden on customers.
And so, AWS has watched the community embed account IDs, regions, and random identifiers into bucket names for years. That must have meant we loved it, because then they shipped that exact pattern as a first-class feature: Account Regional Namespaces.
The feature applies works in that when you create a bucket named myapp-logs and request it in your account-regional namespace: myapp-logs-123456789012-us-east-1-an. The -an suffix signals to the S3 service that this name is scoped to your account and region. Nobody else can register anything-123456789012-us-east-1-an — the 123456789012-us-east-1 segment is reserved for your account. How AWS managed to promise that buckets with an -an suffix don't already exist, and none of those bucket where in a cross-account scenario, is beyond me. Maybe they didn't. The likelihood is very small, that someone already had a bucket with a suffix of -{accountId}-{region}-an, but if they did, and they had a cross account scenario, then that is now broken. Or maybe it isn't, maybe that special bucket according to the new rules was created in the correct account, but in reality someone else owns it.
And so, we can see the same problematic pattern with this one as all the other hacks.
It is opt-in. You must set a special header or use a special property on CreateBucket. Existing buckets are not migrated. Existing tooling does not generate these names. Every piece of infrastructure code that creates S3 buckets needs to be updated to use the new naming convention. And that means every service, SDK, API, library, product, etc... that you are using must also make this change.
It wastes 26+ characters to your bucket name. S3 bucket names have a 63-character limit. You now have at most 37 characters to work with before you hit the wall. If you have a naming convention like {environment}-{team}-{service}-{purpose}, you are already in trouble. Hopefully each team in your organization has their own AWS account, but I know some of us aren't that lucky. You might be asking yourself, why 63? Well this limitation also almost certainly exists because the bucket name has to be part of the url as a subdomain. And DNS parts max out at 63 according to RFC 1123.
It does not address the actual architectural problem. Your bucket is still globally addressable via s3.amazonaws.com. The access model is unchanged. The public bucket problem is unchanged.
And then there is the SDK story.
Clever engineers will immediately ask:
If my bucket name no longer explicitly includes my account ID and region, I cannot just pass around the bucket name. How do I write portable infrastructure?
My answer: You don't.
The obvious AWS's answer: pass the account ID and region as a special token that the SDK resolves at runtime from the current execution environment. Instead of hardcoding 123456789012, you reference a variable that CloudFormation or the SDK resolves from the execution context.
So it's a second hack layered on top of the first one. The question is philosophical but practical, and AWS' answer is technical. That's a weird take.
You now have infrastructure code that creates bucket names by concatenating a prefix with a runtime-resolved account ID and region. Your IaC state needs to capture the resolved name, not the template. Your references to the bucket in other services need to either embed the same resolution logic or accept the full resolved name as an input. Your cross-account pipelines — CI/CD systems deploying into multiple accounts — need to be aware of this resolution mechanism.
AWS did not fix the problem. They added an opt-in feature that partially addresses one symptom, then added tooling to work around the limitations of that feature. You'll notice in the same release post, they also include the changes they had to make to CloudFormation S3 Resource. The people celebrating are celebrating a band-aid on a fracture.
How S3 Is Actually Used
But the real goal of tis article is actually talk about a solution. And to do that we need to review the fundamental use cases of S3. In practice it exists for four distinct use cases. Which of course have almost nothing in common:
1. Private object storage — build artifacts, backups, data lakes, Lambda packages, database snapshots, OpenTofu, Terraform, IaC state files, and SPA access by CloudFront. No direct external access. Internal AWS service-to-service or IAM-authenticated only. I'm go out on a limb and say this is 99% percent of the S3 usage by volume and by bucket count.
2. Event-driven processing — S3 event notifications triggering Lambda functions. An object is created or deleted; an event fires; a Lambda processes it. (One caveat here is that you MUST Never do this because S3 event notifications are not durable, ensure that all S3 events are sent directly to SQS, and then to Lambda.) The bucket name and ARN arrive in the event payload:
{
"Records": [
{
"eventSource": "aws:s3",
"awsRegion": "us-east-1",
"eventTime": "2024-03-01T12:00:00.000Z",
"eventName": "ObjectCreated:Put",
"userIdentity": {
"principalId": "AWS:AROAEXAMPLEID:session"
},
"responseElements": {
"x-amz-request-id": "EXAMPLE123456789",
},
"s3": {
"s3SchemaVersion": "1.0",
"configurationId": "upload-processor-trigger",
"bucket": {
"name": "my-app-uploads",
"ownerIdentity": {
"principalId": "AEXAMPLEOWNERID"
},
"arn": "arn:aws:s3:::my-app-uploads"
},
"object": {
"key": "uploads/photo.jpg",
"size": 1024,
"eTag": "d41d8cd98f00b204e9800998ecf8427e",
"sequencer": "0A1B2C3D4E5F678901"
}
}
}
]
}
Notice what is not in this payload: a public-facing URL. The bucket.name and bucket.arn reference the internal bucket name. S3 ARNs have never included an account ID or region — arn:aws:s3:::my-app-uploads, not arn:aws:s3:us-east-1:123456789012:my-app-uploads. The identifier in the event is already the private bucket identifier, not a public one. And it would be easy to add the region and account ID to this ARN and likely not break a single thing.
And that's the tell. The event-driven use case has always operated on private identifiers. The Lambda function receiving this event doesn't care what the bucket is called publicly, or whether it has a public URL at all. It cares about the object key and the internal bucket reference — both of which are already account-scoped and private by nature. S3's internal event system was already operating on the right model. The global namespace was never part of this path.
3. Presigned URLs — assets that could be served over CloudFront because they are cacheable, but because you don't want them to be public, such as user owned data, you create a strategy to serve user data directly from S3. And same goes in reverse, you allow users to upload data, but rather than needing to deal with it in your service API, you directly have the client integrate with S3.
4. Direct public access — open buckets, bucket website hosting, ACL-public objects, resolvable by a public DNS. This is the pattern that causes all the breaches, all the confusion, and almost all of the architectural complexity AWS has accumulated in S3 over the years.
Category 4 is a tiny fraction of actual S3 usage by any metric you choose. It is responsible for a disproportionate fraction of the design surface area, the security incidents, and the policy complexity. And all the fixes so far make the usages of (1), (2), and (3) more challenging, while increasing the safety of (4). This is not how you solve architectural problems. You want to play a strategy where the most frequent uses are optimized for security, where the threat model identifies the biggest risk, to subvert that, not protect a screendoor or a fence in the middle of the desert.
The data breaches you read about were almost always S3 misconfiguration involving category 4. A few illustrative examples from a single year — 2017 alone:
- Verizon — 14 million customer records including names, addresses, and account PINs, left in a publicly accessible bucket by a third-party vendor (NICE Systems). The bucket was open for weeks after Verizon was notified.
- Accenture — Four public buckets containing 137GB of internal data: credentials, decryption keys, the master AWS KMS access key for their cloud platform, and data from clients across the Fortune 500.
- WWE — 3 million fan records including home addresses, ages of children, ethnicity, and account details. Open to anyone with the URL.
- GoDaddy — Configuration data for 31,000 GoDaddy servers exposed in a public bucket. In a detail that should give everyone pause: the bucket was used and misconfigured by an AWS employee.
The fix in every case should have been "make S3 harder to misconfigure." But the advice and resolution we've seen was instead: "fix your IAM policies", "enable Block Public Access", "audit your bucket ACLs." Patches. Tooling. Guardrails, Security Hub findings around a footgun that should not exist in the first place.
The reason category 4 exists at all is historical. In 2006, if you wanted to serve a file publicly from the internet, you needed a publicly accessible server. S3 was that server. CloudFront did not launch until 2008. IAM did not launch until 2011. The access model AWS ships with S3 today is the access model from an era when the alternatives did not exist yet. (I'm of course speculating here, because I didn't use AWS until 2008, and couldn't find a great source for this.)
Yet, some of the hacks to fix this problem have happened much later than 2011, and realistically, none of them even required IAM to make this happen.
The Real Root Cause
All of that complexity — ACLs, Object Ownership, Block Public Access, website hosting, and the hacks added attempt to fix secord-order mistakes. They were pilled ontop of the one thing nobody touched: the naming model. And it's the real feature everyone wants:
Feature 1: The same logical bucket name across multiple AWS accounts.
Take OpenTofu (or any IaC for that matter) for instance. You need remote state storage. The canonical setup: one S3 bucket per account, typically named something like {org}-opentofu-state or {account-name}-tfstate. Simple, readable, deterministic.
In practice, you have a dev account, a staging account, a production account, a security account, a shared-services account. You want 123456798012-opentofu-state in all of them. Under the current global namespace, you cannot have that. You have to name them 123456798012-opentofu-state-dev, 123456798012-opentofu-state-prod, and so on — encoding the account into the name because the namespace doesn't do it for you.
With the new account-regional namespaces, you can now have opentofu-state scoped to each account. In theory. But in practice, all the changed was the interface for creating buckets, the usage of the buckets and their names are still the same as without this latest feature, and worse, without changing anything regarding how the service actually works, now everyone needs to make change. It is the worst of all fates:
- OpenTofu's and other IaC's S3 backend configuration needs to be updated to use the new naming scheme
- Any modules that reference this bucket by name need to be updated
- Any existing state files pointing to the old bucket names need to be migrated
- Your bootstrap process — the code that creates the state bucket before OpenTofu can run — needs to support the new
CreateBucketheader
None of this is easily managed. And while you can opt out of things like (2) and (3), you all know that there is some "security theater" going on at large enterprises that will claim a migration here "increases security". I'm sure there the associated security hub finding that is going to come out soon with a Critical level. All of it is work that should not have been necessary if the architecture had been correct from the start.
Feature 2: The same logical bucket name across multiple regions.
Multi-region active-active deployments are increasingly common. You want my-app-assets in us-east-1 and eu-west-1. Under the account-regional namespace, these would be my-app-assets-123456789012-us-east-1-an and my-app-assets-123456789012-eu-west-1-an — different names for logically identical resources. Your infrastructure code must now either parameterize the region or generate the full resolved name in every place that references the bucket.
This is the same problem that existed before the fix. The namespace is account-regional — it scopes names to an account and a region. That is correct for preventing name collisions, but it means your logical bucket name is still not portable across regions. The same bucket in a different region is a different name. Your replication configuration, your CDN origin setup, your cross-region failover logic — all of it must carry the full resolved name around. You can have the same DynamoDB Table Name used in every region, but not S3.
The underlying issue is that S3 conflated four separate concerns:
- Identity — what is this bucket called?
- Location — which account owns it, and which region holds the data?
- Addressability — how do external clients find it?
- Accessibility — Who should have access to it?
AWS's new feature embeds all four into the name string itself: myapp-123456789012-us-east-1-an. The account ID is in the name. The region is in the name. The identity is whatever is left over after you subtract those 26+ characters. The an limits access. This is not a namespace — it is a naming convention that happens to be enforced by the S3 service on creation only. The four concerns are still coupled; they are just coupled inside the string rather than explicitly as configuration.
Intelligent Design
I want to be clear, AWS S3 is a fantastic service. It is so great in fact that there are no small number of huge businesses built around duplicating the S3 API. There are 20 years of successes after all. And I don't want to gloss over that:
What S3 gets right
Object storage is the correct primitive. An opaque key — a bucket name and an object path — maps to a sequence of bytes. Durable, versioned, regionally placed, with a consistent API surface across every SDK AWS ships. Lifecycle rules, replication, object tagging, multipart uploads, and locking (but only recently unfortunately). These are the right tools for managing data at scale, and they work.
Additionally, Presigned URLs are the correct mechanism for temporary access delegation. Credential-scoped, time-limited, no IAM policy change required. The object stays private; the URL grants access for a window. That's also the right design.
Do I need to mention the high durability of 99.999999999%, and the reliability of 99.99% as well?
None of this needs to change. The problem isn't storage. It's two things piled on top of storage: the naming model and the access model.
Secure by default
Every AWS primitive designed with security in mind starts from the same position: the unconfigured state is safe.
IAM: default deny on everything. No permission exists until you create one explicitly. The account with no IAM policies grants access to nothing.
VPC Security Groups: inbound traffic blocked by default. Every allow rule is explicit. The security group you just created, without touching it? It denies everything. (excluding the default VPC, which I'm not going to get into here)
KMS customer-managed keys: a key with no resource policy grants decryption to nobody — except the account root, which is a recovery mechanism, not an access path. Grants are explicit.
S3 is the exception.
Secure by default doesn't mean "safe unless you misconfigure it." It means safe by construction. The state you reach without doing anything must be the safe state. And for me that also excludes the presence of pits of failure. If it is easy to do the wrong thing, then this a dangerous state. Public access for instance, must require deliberate, explicit, named work. Not the absence of a flag. Not the absence of a policy. Not a default you forgot to change.
S3 had it backwards. And the fix isn't more flags. The fix is a model where a public bucket cannot exist — because public access isn't a property a bucket can have, it's a property of a feature called "promotion".
My Prospal: Private by Default, Public by Promotion
Here is the core insight that AWS released but no one wanted to commit to:
- Bucket names are global (partly addressed by the new feature, but only for new buckets, only opt-in, only with a 26-character tax)
- Buckets are the unit of access control
- Public access is a property of the bucket
- Anyone with the bucket name and the right IAM permissions (or no permissions required, if it's public) can read objects
The right model: A Private Bucket Service. If you tilt your head sideways and squint, you might see that such a thing has been here all along, and I'm sure there is even an already existing AWS primative that encapsulates this concept internally.
info
By Private , I mean that the bucket is private to your account, not private in the fact that it just isn't publicly accessible.
- Allow the creation of S3 Private Buckets the same way you would the current S3 Public Buckets. Might as well rename the current API to be
Public Buckets Serviceinstead, although I guess PBS was already taken, not to mention Public and Private both start withPa bit of an oversight in the english language. - Private Buckets only exist in that one region in that one account, and make use of the AWS ARNs correctly with aws account ID and region in the ARN.
- All interactions within the account will assume the private bucket, and never the public bucket. These are your API calls through SDKs, Event Source Mappings for SQS, Event notifications.
- Names follow the same strategy as they do today, (although since they aren't public, please let us have upper case characters)
- Objects are private. Not by default. Always. Without exception.
- Public access is not a property of the bucket. (Want to create a public bucket still? I'll get to that in moment.)
I don't think this a novel concept. DynamoDB works exactly this way.
And under this model, my-app-assets in us-east-1 and my-app-assets in eu-west-1 are two separate buckets each globally identifiable via the ARN, and accessible via the region based parameter in the SDK/CLI/API (which by the way is already necessary.) Your infrastructure code references the bucket name as it always has done.
What's missing you might ask?
No 26-character suffix. No runtime SDK token substitution. No encoding of internal topology into names that humans have to read and type. No weird public configuration, no ACLs, no URLs associated with the buckets, no pits of failure.
Public Buckets: How promotion works
A bucket, once created, is private. The bucket's access state never changes. What changes is what you attach to it.
There are two core public scenarios that I'll call promotion paths that still must have solutions for:
Presigned URLs: Temporary Promotion
You issue a time-limited, credential-signed URL for a specific object. The URL encodes the object path, an expiration, and a signature derived from your IAM credentials. Anyone with that URL can read that object — for the duration you specified. When it expires, access ends. The bucket policy didn't change. The object's access model didn't change. The credential signed the request; the routing table resolved the bucket; S3 validated the signature and served the object.
A presigned URL today looks like https://mybucket.s3.amazonaws.com/file.png?X-Amz-Credential=AKID123%2F20240101%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=.... The X-Amz-Credential field already contains the account identifier — derived from the access key ID, which maps to an account. S3 extracts that account, consults the routing table for mybucket in that account, and routes to the right physical bucket. The global uniqueness constraint was never doing the routing work here. The credential was.
I want to say that again, presigned urls will still absolutely work out of the box without any changes.
This is because Presigned URLs are not an S3 concept. They're an IAM concept that S3 validates. To explain, we need to dive into how AWS IAM actually works. AWS IAM uses their custom SigV4 signature strategy for every request to AWS. And every request to AWS goes over the wire on a AWS owned DNS url for the service with all the necessary parameters.
For instance, your SDK computes a SigV4 signature using your IAM credentials — the access key ID and its corresponding secret. No AWS API call is made. The URL is computed entirely locally. This is how it works for every AWS service API. When you call DynamoDB this happens, and the same thing happens when you call S3.
Presigned S3 is a trick. After constructing the full HTTP payload to send to the service, instead of actually sending it, you give it to someone else. Then that person executes the payload. Normally it wouldn't matter who executes it, but what if some part of the payload was allowed to change between the generation of the HTTP payload and the exector executing, let's say for instance: the Binary Body. In this way, you could generate a request that encodes the bucket, the object path, the expiration, and the signature, and hand it to some other user. They present it to S3 with a custom binary.
When S3 receives the request, it extracts the access key ID from X-Amz-Credential, looks up the corresponding IAM entity via STS, re-derives the expected signature, and checks that it matches. Then it checks the expiration. Then it checks that the IAM entity had s3:GetObject permission at signing time. If all three pass, S3 serves the object (or persists it in the case of s3:PutObject).
That's all. S3 is just doing IAM validation, the same thing every other service is doing. It is not checking whether the bucket is public. It is not consulting the access model at all. A fully private bucket — no ACLs, no public access configuration, nothing — can serve objects via presigned URL because the authorization is credential-based, IAM-based, AWS-API based, it is not a unique access-model built into public S3-based buckets.
Public Buckets: Permanent Promotion
Since, public access is not a PrivateBucket property, there has to be some way to still expose public access to the PrivateBucket data. And so the proposal would allow making a PrivateBucket public by requesting a bucket name from the global authoritative S3 Bucket Name list. The same process you already have today for S3 buckets, when you create a new one.
In the new model, the public properties, the ACLs, the website configuration, aren't properties of the bucket. They're a separate resource: a public access configuration. Which today is what is called S3. So you might be able to see why I'm suggesting a name change. When you create one, attach it to your private bucket, and the S3 URL is created. You remove it, and the S3 URL stops existing. The bucket itself never changes state. The URL is a consequence of the configuration, not a property of the storage. And that URL, that's the thing that must be globally unique, and most importantly that doesn't even need to match the original bucket, and it won't.
Website configuration lives there too. Index documents, error documents, redirect rules — these move from bucket settings into the public access configuration. The s3-website endpoint exists because the configuration says it should, not because the bucket was created with a flag set.
And because the user-defined string — the bucket name — is preserved through the Public Bucket configuration. What is no longer true is that this string must be globally unique for the private bucket. That constraint was never load-bearing. It was just there because of the expectation on public usage.
The Custom HTTP domains using S3 website hosting — with CNAMEs pointing to mybucket.s3-website-us-east-1.amazonaws.com or not — continue to work. The website configuration moves into the public access configuration resource; the s3-website endpoint continues to exist as long as that configuration exists. No customer change is required.
Because this functionality is separate, AWS can disable (and hopefully dismantle) in one huge swath all of the public features of S3 that are insecure by default, and lead new AWS accounts down the path of CloudFront for public access. If you need custom domains, TLS termination on your own domain, caching, WAF, HTTP/2, geographic restrictions, or edge functions — that's not an S3 question. That's a CDN question. And the answer is CloudFront as the reverse proxy with a private S3 bucket origin granted access via the Origin Access Control configuration.
The bucket stays private. CloudFront has authorized access to it. Your users get a production-grade delivery layer with every security consideration you need. S3's job is to hold the bytes and serve them to one authenticated caller — the distribution. CloudFront's job is to serve those bytes to the world under your domain, your TLS certificate, your cache rules.
This is already how every serious production setup works. The new model doesn't change that. It just makes it the only coherent option, instead of one option among several confusing ones.
A New CloudFront Opportunity
Presigned URLs have a structural limitation today that nobody talks about: the SigV4 signature is computed over the canonical request, which includes the Host header. And so the URL is signed against mybucket.s3.amazonaws.com. Change the hostname and the signature fails. Which actually is a huge problem for CloudFront Functions when rerouting requests to a different origin (sometimes it works). This means custom domains for presigned URLs are impossible today. Every download link, every document export, every profile photo URL your product generates contains s3.amazonaws.com. Your customers see your infrastructure provider in every URL. There is no way around it with the current model.
The right fix is for CloudFront to gain first-class presigned URL support: the ability to validate SigV4 signatures on behalf of S3. If CloudFront can validate the signature, the URL can be generated against your CloudFront custom domain — with your ACM certificate, on your domain — and CloudFront handles the validation and the downstream request to S3. The signing mechanism doesn't change. The client code doesn't change. The SDK GeneratePresignedURL call works identically, just against a different hostname. Ironically, CloudFront offers some partial functionality for Signed Request URLs and Signed Cookies, but these actually have a security hole because they don't include the same level of control that IAM policies provide. CloudFront + IAM would be a real game changer for Presigned URLs.
The S3 team's outstanding task
Now on to easy but annoying part. AWS cannot simply remove public bucket support creation path. It isn't the millions of buckets in production, but rather all the code paths that create buckets and then make assumptions about them. Some of those code paths were written by teams that no longer exist.
Any migration strategy that requires customers to take action will fail for the long run. The path forward has to be one where the default behavior improves without requiring every customer to update their infrastructure. Something that the current history of hacks haven't gotten correct at all. (Although their folly resulted only in decreased security rather than broken configuration.)
AWS can either trudge along with this currently broken S3 architecture riddled with pits of failures. Or they can admit they made a mistake and default all new accounts' buckets to not contain a public access strategy. This is actually the right thing to do, and they can do this safely as they have deprecated even whole AWS services before.
Phase 1 — New accounts, new defaults
- Public S3 Buckets completely disabled by default, no website hosting, no ACLs, no bucket policies. All of these are blocked from usage without a support ticket. We don't need the public configuration.
This doesn't break existing buckets. And new infrastructure gets the right defaults. The blast radius is almost zero. There are some AWS organizations out there that are dynamically creating S3 buckets in automatically provisioned new AWS accounts with assumptions based on how buckets work. When creating a new account and then a bucket in that account, they will see a problem. This just needs to be communicated.
You might be thinking, couldn't there just be a magic flag on bucket creation that specifies that the bucket is account/region bound, call that flag: private: true. The problem is removing the restriction to private buckets MUST BE OPT-OUT. private: true makes the default the legacy insecure current state, and keeps public access is opt-out. And therefore it still allows all the bucket negligence awards that Corey is so keen on giving out. A flag is not sufficient, and instead there needs to be a mature approach to the migration. Which is why the recommendation here is:
- Rename S3 everywhere to "S3 Public Bucket Configuration"
- Reintroduce S3 as a Private Bucket concept
Phase 2 — AWS internal service updates
AWS has some internal work to do. Luckily most of the mess that was caused is squarely cornered into the S3 Public Bucket Configuration and none of it actually affects our new private bucket creation or usage. That means, after the rename, AWS can go back through all of their services and retarget all interactions with S3 to use the new Private S3 SDKs/API. This is squarely in their control.
S3 Bucket Events + Lambda Event Source Mapping
One area where there is a bit of a cross over are events like S3 Events over SQS => Lambda. But as discussed earlier that's actually a no-op. Similarly, Lambda Event Source Mapping (Lambda ESM), used for automatically polling SQS is a non-issue. But the reason why is worth understanding. An ESM configuration is account-scoped in the first place. When you set up a Lambda trigger, you're making an authenticated API call inside your account: "Lambda function X should fire on events bucket Y." The ESM record lives in your account. The bucket lives in your account. AWS resolves the bucket reference using the account context of that API call — not the public namespace.
The current ESM ARN looks like arn:aws:s3:::mybucket — no account ID, no region, because those were implicit in the global uniqueness guarantee. In the new model, mybucket is a private identifier scoped to your account. The ARN format doesn't change. The resolution just shifts from "global name lookup" to "private identifier lookup within account context" — which AWS handles internally. No customer touches their ESM configuration. No ARN format changes. No trigger reconfiguration. Future ARN formats for the ESM should take the account ID and the bucket region, but AWS needs to maintain the global mapping table they already have that allows the account-less, region-less ESM bucket ARN to resolve the bucket in the specific region, in the correct specific account. In other words, ESM resource should accept either the global bucket naming strategy or the region-account local one.
The message here "Update your Event Source Mappings for Buckets so that you have the account ID or region specificed". This might be the first ever [Action Required] email, that actually has a required action. Or maybe they'll just update Security Hub to include a finding to fix this, and an AWS Config rule that validates it with an automatic remediation.
CloudFront S3 origin compatibility
CloudFront is not part of S3's access model — it's a CDN that sits in front of a private S3 bucket, authorized via OAC. That already works today and obviously must continue to work in the new model. The only S3-specific change AWS needs to make is ensuring that CloudFront's S3 origin configuration resolves bucket references using the private identifier rather than the global name. Again, that is an internal AWS concern. No customer CloudFront configuration changes. I'm sure there is someone out there that is going to request cloudfront have access to S3 buckets in another account. AWS can easily support a similar solution to the ESM as above, CloudFront accepts either the global S3 ARN or the account-region localized one.
Phase 3 — Configuration Split
Every existing S3 bucket is already the private half of the new model. Customers haven't been creating "public buckets" — they've been creating private buckets and then attaching public configuration to them in the form of ACLs, Block Public Access exemptions, Bucket Policies, and website hosting settings. The private bucket has always existed. What hasn't existed is the explicit separation exposed to AWS Account users. That starts now.
Since the buckets themselves and the public access configuration don't actually change here, the only thing AWS has to do is backpopulate a list of S3 private buckets whose names will be the exact same same as the current PublicBucket name. The goal being that all AWS S3 buckets should be referencable by their account-region localized arn, and the relevant console UI exists to display that. That's a script even Kiro could write in an afternoon.
Presigned URL configuration handling
As argued above, the Presigned URL configuration already will work out of the box since the exact same problem has already been solved for literally every other resource in AWS. The one caveat here is that there will likely need to be a new method GeneratePresignedBucketUrlForPrivateBucket to make sure it includes the account Id and the region explicitly so that the public bucket configuration isn't necessary to continue to use that option. That's because the current method doesn't take in the account ID or the region, but just the bucket name.
The one exception is cross-account presigned URLs — an IAM identity in Account B generating URLs for a bucket that lives in Account A. I personally don't even know if this is possible, but technically I don't see why not. In this case, if we use the X-Amz-Credential to determine the account, AWS would incorrectly assume the account is B (where the identity is) and not Account A (where the bucket actually lives). But AWS S3 have very competent architects, so I'll leave that challenge for them to solve (I can imagine using this same new GeneratePresigned menthod I just suggested above).
It's also worth noting that potentially the presigned URL configuration could be an explicit resource you create when you need it similar to the public access. And by default just create it for all existing buckets.
Phase 3 — Deprecation
The best part of this design is that regarding deprecations there are none! Since all we are actually doing is changing the same of some SDKs to improve readibily and really just the text in the UI. The only real change that is necessary here is going through all the docs and updating the content with more appropriate and clear naming.
Most importantly, over time, the "public bucket" moniker will disappear entirely from the documentation as a concept, from customer usages, and most importantly from the news. And what replaces it? A private bucket with an explicit access configuration attached when needed. Two resources, two concerns, neither coupled to the other by default. The access model that caused two decades of breaches stops being something new engineers get to learn about.
The Objections
Proposing a fundamental redesign of S3's control plane will attract objections. Here are the ones I felt like addressing:
What About SPA Websites?
The most common objection: "But I host my react/vue/solidjs app on S3 with website hosting enabled, and it works fine."
It works, but it isn't correct architecture. Let's be precise about what is actually happening.
Your S3 bucket is serving HTTP at http://my-app.s3-website-us-east-1.amazonaws.com. Your domain is resolved by one two ways:
Option A — CNAME directly to the S3 website endpoint. — You have no TLS. S3 website hosting is HTTP only — it has no mechanism to serve HTTPS for a custom domain. Your users therefore must be on HTTP, so this is not a viable production setup. It actually doesn't work at all.
Option B — CloudFront in front. — CloudFront handles TLS (via ACM), your custom domain, HTTP→HTTPS redirects, the 404 → /index.html behavior for client-side routing, cache headers, compression, and geographic distribution. S3 is behind CloudFront, serving bytes when requested.
Option C — The website domain is the S3 url — You are freely passing out your S3 bucket URL to clients and asking them to remember that custom url. Something for sure is going to break some day, but nothing stopped you from doing it.
Sites that only use S3 website hosting without CloudFront, serving plain HTTP is not a counterexample. It is a site that is broken and getting more broken by the day. Chrome announced in 2023 that it is moving towards HTTPS by default, automatically upgrading HTTP navigations to HTTPS. An S3 website serving HTTP gets upgraded to HTTPS by the browser, and since S3 cannot serve HTTPS on a custom domain, the request fails. Firefox has had an HTTPS-Only Mode available since 2020 that blocks HTTP sites entirely. These are not future concerns. They are not esoteric. They are not nuanced. They are the current state of the web. A site that only works over HTTP is not a production website in 2026. It is a broken website that has not been maintained.
Which means in every functional production scenario, S3 website hosting is doing nothing useful. CloudFront is handling everything. S3 is holding bytes.
Therefore, Option B is every production SPA, S3 website hosting is contributing nothing. CloudFront is doing all the work that makes the setup viable. The bucket does not need to be public. Website hosting does not need to be enabled. The only reason engineers enable website hosting is that they are following a tutorial that predates CloudFront's ability to serve private S3 buckets, and nobody told them the tutorial was outdated. Or more likely, someone did, but they didn't listen.
CloudFront likely has been able to serve our new Private S3 bucket concept since Origin Access Control (OAC) replaced the older Origin Access Identity (OAI) approach. OAC supports server-side encrypted buckets, covers all S3 regions, and signs requests to private S3 using SigV4. Even before OAC, your bucket never needed to be public.
There could be a concern that CloudFront doesn't know how to talk to anything other than a public S3 bucket or a public URL. But interestingly enough, CloudFront now also supports private origins via ALB with VPC origins, which closes the last remaining scenario where direct public exposure might have been argued as necessary. You can run your origin entirely inside a VPC, with no public exposure, and serve it through CloudFront. The gap is gone.
And the "CloudFront costs more" objection doesn't land either. CloudFront has a free tier: 1 TB of data transfer per month, 10 million HTTP requests, and 2 million CloudFront function invocations. A landing page or documentation site that fits in an S3 bucket almost certainly fits within that free tier, and even if it doesn't, at scale you are still getting the benefit of the cost reduction.
A complexity argument would be more interesting. Setting up a CloudFront distribution requires more steps than enabling S3 website hosting. That is true. But the complexity exists either way, it is just hidden. And you still need TLS. You still need the index.html routing behavior for client-side routing (or a more expensive CloudFront function). You still end up at CloudFront. The engineers who skip it are the ones serving HTTP from a subdomain with no TLS, which is screams for a denial-of-wallet attack.
And for users who genuinely have not set up CloudFront, a la Option C : the AWS S3 migration plan already answers this. The configuration split means existing public buckets keep their public access configuration intact, those sites keep working. The owner does nothing. When they are ready to do it correctly, the options are available.
Bucket Origin Responses
There is one thing I left out, and I didn't want to bring this up because it's annoying, but I'm sure someone will call me out on it.
When you set up S3 as an origin for your CloudFront, you might have the need to control the response headers. Historically, you were not able to configure anything in CloudFront, let alone do it dynamically. And so using S3 to set the CORS policies or other security policies was required. However now, CloudFront offers response headers, and while it isn't everything, even S3 isn't sufficient for specifying all the relevant headers. While I don't love it, for Authress, we have a CloudFront Function attached to every response. There is a performance hit and a cost hit to do this on literally every S3 related request. But argubly it is a small price to pay to have CloudFront do the thing that it should be doing all along, and not to save this configuration in S3 where it doesn't. Maybe AWS could be nice and still offer this configuration in S3, or be nice and add this as an option to CloudFront, or be nice and make CloudFront functions even cheaper, because why not, API Gateway velocity templates are free after all!
You're asking AWS to blow up a working control plane
Yes. That is what a migration looks like. The alternative is two more decades of incremental patches, each one adding more surface area and more documentation burden without touching the underlying design, and worst of all, still enables a massive pit of failure.
The control plane does not need to be blown up for customers. The translation layer proposal in the previous section means existing workloads continue working. What needs to change is the model exposed to new infrastructure — the primitives developers learn, the defaults they encounter, and the architecture that tutorials recommend.
AWS has done this before. The IAM role model replaced key-based authentication for most AWS-to-AWS access patterns. And AWS IIC replaces IAM roles for organizations and SSO. CloudFront Origin Access Control replaced Origin Access Identity. Neither replacement was instantaneous, and neither broke existing workloads. The old model continued working through a maintained compatibility layer while the new model became the default for anything new.
The objection treats "existing behavior must never change" and "defaults must never improve" as the same thing. They are not.
The Better Announcement
The Account Regional Namespaces announcement solves one real problem, the name collisions, using an opt-in mechanism with a 26-character tax on your bucket names, tooling that requires SDK and CloudFormation updates to remain portable. But it has zero impact on the access model that causes actual harm.
The right announcement would have looked like this:
- The best feature ever Private Buckets: Account-regional namespaces are the default — for all new bucket creation, no suffix, no opt-in, just the natural behavior that every engineer already wanted is now expected. Change nothing, get all the value.
- The recommendation for public content: A managed CloudFront promotion layer — as the only path to public content, surfaced as a first-class feature with its own console workflow, not a best practice buried in the CloudFront documentation. Because for some reason, AWS likes to improve their console, it still surprises me for how many ClickOps isn't just a migration strategy but a business critical one.
- Backwards compatibility is still and always will work — Legacy ACLs and direct public bucket access still exist — but as of today they are deprecated and require a support ticket to activate. The on-ramp is gone. The escape hatch remains, for now.
Instead, we got a feature that requires you to append -123456789012-us-east-1-an to your bucket names, a second feature that lets your SDK dynamically resolve that suffix from the execution environment, and a wave of blog posts explaining how to wire these two features together. And of course we still have to wait for your-favorite-tool™ to implement this funcitonality.
This is not a fix. It is a patch on top of a patch, with new documentation for how to apply both patches correctly. AWS has a long history of excellent engineering, but I don't concern this new functionality to be part of it.
The gap between "what was shipped" and "what would fix the problem" is not subtle. It is not a matter of resources or engineering difficulty. Name collisions, the problem I can only imagine customers have been filing tickets about for years, was partially addressed. But the access model that still will cause actual harm was not.
Until the access model changes, the endless stream of conflicting advice will remain out there on the internet.
For help understanding this article or how you can implement auth
and similar security architectures in your services, feel free to
reach out to us via the community server.




Top comments (0)