Same AWS Error, Two Answers: I Built a Tool That Stops Guessing at IAM

Ciao Amici 👋

Let me start with a confession. Every time I hit an AccessDenied in AWS, I do the same lazy thing you do. I copy the wall of text, paste it into a chat model, and ask "why?".

And every time, I get back a tidy little list of maybes. Maybe it is your identity policy. Maybe it is an SCP. Maybe a permissions boundary. Maybe a resource policy. All plausible. All confident. None of them actually checked against my account.

That bothered me for a long time. The model was not lying to me exactly. It was doing the only thing it could do with the words I gave it. But "here are five things it might be" is not an answer. It is a starting point for an afternoon of clicking through the IAM console.

So I built a tool that stops guessing and starts checking. It is called Rosetta, and this post is the full story of building it, including the bug I found in my own ranking logic and the Bedrock wall I walked straight into.

Here is the whole thing in one screenshot, before I explain a single line of code.

The money shot

Same error string. One extra flag. Two completely different answers.

First, text only, no credentials at all:

  AWS Rosetta  [text-only]
  code=AccessDenied  op=GetObject  action=s3:GetObject
  principal=arn:aws:iam::896415180455:user/rosetta-victim
  resource=arn:aws:s3:::rosetta-demo-secret/report.txt

  1. [########............] 40%  No identity-based policy allows the action
  2. [########............] 40%  Explicit deny in a Service Control Policy
  3. [########............] 40%  Explicit deny in a permission boundary
  4. [########............] 40%  Resource policy denies or omits the principal
  5. [########............] 40%  Missing KMS grant on an encrypted resource
  note: Text-only mode. Pass a read-only role ARN for authoritative causes.

Five causes. All at 40 percent. The tool is shrugging politely.

Now the same error, with one --role-arn flag pointing at a read only role:

  AWS Rosetta  [enriched]
  code=AccessDenied  op=GetObject  action=s3:GetObject
  principal=arn:aws:iam::896415180455:user/rosetta-victim
  resource=arn:aws:s3:::rosetta-demo-secret/report.txt

  1. [###################.] 96%  Explicit Deny in an attached policy
     fix: An explicit Deny matched: user_rosetta-victim_rosetta-victim-policy.
          Remove or scope down that statement.
     - iam:SimulatePrincipalPolicy (authoritative)

One cause. 96 percent. The exact policy named. The exact statement named. Evidence attached.

That collapse, from five-way uncertainty to one cited answer, is the entire point of this tool. Everything below is how it works and how you can run it against your own account.

Why the guess is not the tool's fault

Here is the uncomfortable truth. From the error text alone, nobody can do better than guessing. Not a model, not a regex, not me.

The string AccessDenied when calling GetObject is genuinely ambiguous. It can mean any of these, and the message often will not tell you which one:

• No Allow in the identity policy
• An explicit Deny in the identity policy
• An explicit Deny in a Service Control Policy above the account
• A permissions boundary that excludes the action
• A resource policy on the bucket that omits the principal
• A missing KMS grant on an encrypted object

Six very different fixes. One identical error message. The words on your screen simply do not carry enough information to choose between them.

Every "paste it into AI" tool tops out right here. It reads the words and pattern matches to the most common causes. That is exactly what my text only mode does, and it is honest about it. Five causes, flat confidence, a note telling you it is guessing.

The fix is not a smarter prompt. You cannot prompt your way to information that is not in the input. The fix is to stop reading the words and start querying the account that produced them.

The turn: SimulatePrincipalPolicy

AWS has one API that answers the question authoritatively: iam:SimulatePrincipalPolicy. You hand it a principal, an action, and a resource, and it evaluates the full policy chain the way IAM itself does at request time. It returns an allow or deny decision, and crucially, the exact statement that made the call, plus separate signals for whether an SCP or a boundary was the thing that blocked you.

This is the heart of Rosetta's enrichment. Here is the real code:

def simulate(session, principal_arn, action, resource):
    if not (principal_arn and action):
        return []
    iam = session.client("iam")
    resp = iam.simulate_principal_policy(
        PolicySourceArn=principal_arn,
        ActionNames=[action],
        ResourceArns=[resource] if resource and resource.startswith("arn:aws") else ["*"],
    )

    out = []
    for r in resp.get("EvaluationResults", []):
        detail = {
            "action": r.get("EvalActionName"),
            "decision": r.get("EvalDecision"),
            "allowed_by_organizations": r.get("OrganizationsDecisionDetail", {}).get("AllowedByOrganizations"),
            "allowed_by_boundary": r.get("PermissionsBoundaryDecisionDetail", {}).get("AllowedByPermissionsBoundary"),
            "matched_statements": [m.get("SourcePolicyId") for m in r.get("MatchedStatements", [])],
        }
        out.append(Evidence(
            kind="simulate",
            source="iam:SimulatePrincipalPolicy",
            detail=json.dumps(detail),
            authoritative=True,
        ))
    return out

Notice what comes back. Not just explicitDeny, but allowed_by_organizations and allowed_by_boundary as separate booleans. That is how Rosetta distinguishes an SCP deny from a boundary deny from a plain policy deny, three completely different fixes that share one error message. The API already knows all of this. My job was just to read it correctly, which, spoiler, I did not do on the first try.

The three fields work like a decision tree. If allowed_by_organizations is false, an SCP is your problem and no amount of editing the user's policy will help. If allowed_by_boundary is false, the permissions boundary is clamping you. Otherwise the decision field tells you whether it was an explicit deny or simply the absence of any allow. Rosetta turns each of those into a distinct, correctly worded cause.

The architecture, in four moves

Rosetta is a pipeline. Each stage is a small pure function, which meant I could test the whole thing without touching AWS until the very end.

parse    error string -> structured fields
enrich   assume read-only role, run simulate, fetch resource policy, match CloudTrail
reason   optional Bedrock pass over the evidence
rank     authoritative evidence beats model guesses beats corpus fallback

The design decision I am happiest with is the hybrid model. Text only mode needs zero credentials and runs anywhere, which makes the tool useful even when you are looking at someone else's pasted error with no access to their account. Enrichment needs exactly one thing: a read only role you let Rosetta assume. Nothing else. Your credentials never leave your machine, and the role only ever has read permissions.

The parser is boring regex, and that is fine. Boring is reliable:

_CODE = re.compile(r"An error occurred \(([A-Za-z0-9.]+)\)")
_OP = re.compile(r"when calling the (\w+) operation")
_PRINCIPAL = re.compile(r"(?:User|Role):\s*(arn:aws[^\s]+)")
_ACTION = re.compile(r"perform:\s*([a-zA-Z0-9]+:[A-Za-z0-9*]+)")
_RESOURCE = re.compile(r"on resource:\s*(arn:aws[^\s]+|\*|[^\s]+)")

Feed it a botocore error string, get back a clean struct with the service, action, principal, and resource pulled out. That struct is the contract every downstream stage depends on. If parsing gets the principal ARN wrong, the simulate targets the wrong identity, so this unglamorous regex is load bearing.

Setting up the read only role

The authoritative mode needs a role Rosetta can assume, and that role needs exactly the read permissions the enrichment uses and nothing more. Here is the policy, straight from the repo at infra/readonly-role-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RosettaSimulate",
      "Effect": "Allow",
      "Action": [
        "iam:SimulatePrincipalPolicy",
        "iam:GetPolicy",
        "iam:GetPolicyVersion",
        "iam:ListAttachedRolePolicies",
        "iam:ListAttachedUserPolicies"
      ],
      "Resource": "*"
    },
    {
      "Sid": "RosettaResourcePolicies",
      "Effect": "Allow",
      "Action": ["s3:GetBucketPolicy", "kms:GetKeyPolicy"],
      "Resource": "*"
    },
    {
      "Sid": "RosettaCloudTrail",
      "Effect": "Allow",
      "Action": ["cloudtrail:LookupEvents"],
      "Resource": "*"
    }
  ]
}

Read only, top to bottom. Nothing here can change your account. The role's trust policy points at whichever principal runs Rosetta, your CLI identity locally or a Lambda execution role if you deploy it. Once the role exists, you pass its ARN and Rosetta assumes it with STS, does its read only work, and throws the session away.

Reproduce the demo yourself

I did not want to test enrichment against my real infrastructure, so I built a deliberately broken IAM setup as a throwaway CloudFormation stack. One stack up, one stack down, no leftover mess to hunt for later. The core of it is a victim user carrying an Allow on S3 reads, then an explicit Deny on one secret bucket:

VictimUser:
  Type: AWS::IAM::User
  Properties:
    UserName: rosetta-victim
    Policies:
      - PolicyName: rosetta-victim-policy
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Sid: AllowS3Read
              Effect: Allow
              Action: s3:GetObject
              Resource: "*"
            - Sid: DenySecretBucket
              Effect: Deny
              Action: s3:GetObject
              Resource: arn:aws:s3:::rosetta-demo-secret/*

That explicit Deny is the exact thing the money shot catches. Deploy the stack, run Rosetta against it, get your own before and after, then delete the stack and everything vanishes in one action. The full template is in the repo under scenario/.

The one trick worth stealing

If you take one technical thing from this post, take this.

I wanted Rosetta to find the actual failed call in CloudTrail, so it could show you the real request that got denied, with the source IP and the resources involved. My first instinct was to look it up by request ID. That is the natural key, right? Every AWS error has one.

Wrong. CloudTrail's LookupEvents does not accept RequestID as a lookup attribute. The allowed keys are things like EventName, Username, and EventSource. Request ID is simply not on the list, and if you pass it you get a ValidationException.

So the trick is: look up by EventName, pull a small batch of recent events, then filter on the requestID field inside the returned event JSON. The ID is in the payload, just not in the index you can query on.

def cloudtrail_match(session, operation, principal_arn, request_id):
    ct = session.client("cloudtrail")
    resp = ct.lookup_events(
        LookupAttributes=[{"AttributeKey": "EventName", "AttributeValue": operation}],
        MaxResults=20,
    )
    for ev in resp.get("Events", []):
        record = json.loads(ev.get("CloudTrailEvent", "{}"))
        if request_id and record.get("requestID") != request_id:
            continue
        # matched the exact failing call
        ...

One more honest detail. CloudTrail has a delivery delay, so a very fresh error may not have a matching event yet. Rosetta says so rather than inventing one. Pretending you found the event when you did not is worse than admitting the delay.

The bug I found by testing my own tool

Here is the part every reviewer actually cares about, because it is the part most posts quietly leave out.

I pointed Rosetta at my broken stack. The enrichment fired, the simulate ran, and Rosetta confidently told me:

No policy allows this action (implicit deny)

Except it was not an implicit deny. It was a screaming explicit Deny, sitting right there in the policy I had written thirty seconds earlier. My ranking code was wrong.

The cause was dumb and instructive. My rank.py was checking the decision but had brittle branch ordering, and worse, it was hiding the raw simulate decision, so I could not even see what the API had actually returned. The tool was making an authoritative claim while concealing the evidence for it. That is the most dangerous kind of bug, the confident one.

I rewrote the ranking to read the decision properly and to always surface the real EvalDecision and matched statement. The branch that matters now reads:

if decision == "explicitDeny":
    return Cause(
        "Explicit Deny in an attached policy",
        0.96,
        f"An explicit Deny matched: {label}. Remove or scope down that statement.",
        [evidence],
    )

To confirm the fix, I added a --json flag that dumps the raw pipeline output, every piece of evidence included. Running it showed the truth the pretty output had been hiding:

"decision": "explicitDeny",
"matched_statements": ["user_rosetta-victim_rosetta-victim-policy"]

The API had been telling me explicitDeny the whole time. My code was just not listening. The lesson stuck with me: when your tool makes an authoritative claim, make it show its work, or you will end up trusting a bug with a confidence bar next to it.

The wall: Bedrock, Marketplace, and a payment instrument

I wanted an optional LLM layer on top, using Bedrock to phrase the causes more naturally and catch anything my rules missed. This is where AWS taught me something genuinely new.

I picked a Claude model, set my BEDROCK_MODEL_ID, and invoked. Access denied. But not an IAM denial, a Marketplace one:

Model access is denied due to INVALID_PAYMENT_INSTRUMENT:
A valid payment instrument must be provided.

Here is what is going on, because it took me a while to piece together. AWS retired the old manual model access page. Models now auto enable on first invocation, which sounds convenient. But that auto enable quietly creates an AWS Marketplace subscription behind the scenes, and that subscription needs a valid payment method to complete. My account did not have one on file. AWS even emailed me a Marketplace agreement that started and expired at the exact same second, because the billing failed instantly.

This is undocumented tribal knowledge, so let me save you the afternoon I lost: if you get INVALID_PAYMENT_INSTRUMENT from Bedrock, it is not your IAM policy, it is not your region, and it is not the model ID. It is billing. Add a payment method, or use a model that does not route through Marketplace.

I chose to ship without the reasoning layer entirely, and here is why it does not matter one bit. The authoritative answer, the 96 percent explicit deny with the matched statement, comes from SimulatePrincipalPolicy, not from any model. The LLM was only ever going to reword what the deterministic check already knew for certain. Reasoning is a garnish. The simulate is the meal.

So I made the model layer truly optional. No BEDROCK_MODEL_ID set means Rosetta runs parse plus enrich and says nothing at all about missing models:

if os.environ.get("BEDROCK_MODEL_ID"):
    try:
        model_output = reason(parsed, evidence, region=region)
    except Exception as e:
        notes.append(f"Model reasoning unavailable: {e}")
        model_output = {"causes": []}
else:
    model_output = {"causes": []}

Graceful failure is a feature

One more thing the build taught me. After I tore down the test stack, I ran the enriched command again out of habit. Rosetta could no longer assume the role, because the role was gone with the stack. It did not crash. It fell back to text only and told me exactly why:

note: Enrichment skipped: AccessDenied when calling the AssumeRole operation:
      User aws-rosetta is not authorized to perform sts:AssumeRole on
      resource role/rosetta-readonly

That is precisely the behavior you want from a tool that reaches into live accounts. When the privileged path is unavailable, degrade to the safe path and explain the drop in plain language. Never pretend, never crash, never leave the user wondering whether the answer they got was the good one or the fallback. The mode badge at the top, [enriched] or [text-only], tells you which brain answered.

Four ways to run it

The whole tool is one engine with four front doors, so you can reach for whichever fits the moment:

• CLI: aws-decode "<error>" is the primary interface, and the one I use daily.
• Web: web/index.html is a paste box in the same off white and orange I use everywhere. Run python web/local_server.py for a local backend and it works with zero AWS setup.
• API: api/handler.py is a Lambda Function URL handler, ready to deploy with the SAM template in infra/.
• Library: from rosetta import resolve if you want to fold it into something bigger.

All four call the same resolve function, so they cannot drift apart. Fix a bug in the engine and every surface gets it.

A note on cost and safety

Text only mode is free. It makes no AWS calls at all. Enrichment makes a handful of read only API calls per error, SimulatePrincipalPolicy, maybe a GetBucketPolicy, a LookupEvents, all of which are effectively free at any sane volume. The optional Bedrock layer is the only thing that costs money, and it is per token and tiny, and it is off by default.

On safety, the important line is that Rosetta reads your private IAM state, so it is designed to run against your own account with your own role, on your own machine. That is why the product is the repo, not a hosted service. A public endpoint that assumed roles in my account would be a gift to strangers. You run it, you own the credentials, they never leave your laptop.

Try it yourself

Rosetta is open source, MIT licensed, and runs against your own account with your own read only role.

Repo: https://github.com/mursalfk/aws-rosetta

Text only mode needs nothing but Python:

git clone https://github.com/mursalfk/aws-rosetta.git
cd aws-rosetta
python -m venv .venv
source .venv/bin/activate        # Windows: source .venv/Scripts/activate
pip install -e .

aws-decode "An error occurred (AccessDenied) when calling the GetObject operation: ..."

For the authoritative mode, create a read only role from infra/readonly-role-policy.json, trust the principal that runs Rosetta, and pass it with --role-arn. The scenario stack under scenario/ lets you reproduce the exact before and after from this post.

What I actually learned

Four things stuck with me.

Text alone can only ever guess, because the same error genuinely has many causes. The account holds the answer, and one API, SimulatePrincipalPolicy, will hand it to you authoritatively if you bother to ask.

Make your tools show their work. My ranking bug survived exactly until I forced the raw decision into the open with a --json flag. An authoritative claim you cannot inspect is a bug waiting to be trusted.

Degrade loudly, not silently. The moment the privileged path disappears, say so and fall back. A tool that quietly gives you a worse answer without telling you is worse than one that crashes.

And when AWS says INVALID_PAYMENT_INSTRUMENT, go check your billing, not your policies. You are welcome.

Ciao! 👋