DEV Community: Stephen Metcalfe

Authenticated Isn’t Authorized: The AI Code Review Bug That Looks Secure

Stephen Metcalfe — Tue, 25 Aug 2026 21:05:45 +0000

One of the more dangerous mistakes in AI-generated code is also one of the easiest to miss in review. The code has authentication; the user is logged in, and there is an authentication check somewhere in the request path. Everything looks right, but the endpoint still lets one user access another user's data.

The problem is simple:

Authentication tells you who the user is; Authorization tells you whether that user is allowed to access this specific resource.

AI-generated code often gets the first part right and quietly skips the second.

The bug that looks secure

Imagine an endpoint like this:

def show
  return head :unauthorized unless current_user

  @invoice = Invoice.find(params[:id])
  render json: @invoice
end

At first glance, this doesn't look reckless.

The user must be logged in.
The invoice is loaded normally.
There is no obvious injection vulnerability.
The controller is small and readable.

But the important question is missing: Does this invoice belong to the current user?

If I am logged in as user 42 and request:

/invoices/123

The code checks only that I am authenticated: it doesn't check whether invoice 123 is mine.

If changing 123 to 124 gives me somebody else's invoice, authentication has done nothing to protect that data.

This is the class of problem commonly described as BOLA (Broken Object Level Authorization) or, in older terminology, IDOR: object-level access control is missing even though authentication exists.

Why AI is good at producing this bug

AI coding tools are often very good at reproducing familiar application patterns. A controller action often looks like:

@invoice = Invoice.find(params[:id])

An authentication check often looks like:

return head :unauthorized unless current_user

Both are individually plausible. Put them together and the code looks secure.

That is the problem.

The generated code has matched two common patterns without necessarily reasoning about the relationship between the authenticated user and the object being loaded.

The failure is not:

“There is no security.”

The failure is:

“Security exists, but it is not being applied to this resource.”

That is much harder to catch casually.

The safer version

The exact implementation depends on the application, but the data access should normally be constrained by the user's permitted scope.

For example:

def show
  @invoice = current_user.invoices.find(params[:id])
  render json: @invoice
end

Now the lookup itself enforces the ownership boundary.

A request for somebody else's invoice doesn't merely fail an if statement later. The resource is outside the queryable scope in the first place. In a more complex system, that boundary may come from a policy object, tenant scope, permission service, or domain rule instead.

The implementation is less important than the review question:

Can I trace the current user's permission all the way to the specific object being accessed?

If I can't, I treat it as a security finding that needs verification.

Why tests often don't save you

This kind of bug is especially easy to miss when the generated tests were written by the same AI that generated the implementation. You often get tests like:

it "returns an invoice for an authenticated user" do
  sign_in(user)

  get "/invoices/#{invoice.id}"

  expect(response).to have_http_status(:ok)
end

That test passes. It also proves almost nothing about authorization. The test you actually need is closer to:

it "does not allow a user to access another user's invoice" do
  sign_in(user)

  get "/invoices/#{other_users_invoice.id}"

  expect(response).to have_http_status(:not_found)
end

or whatever denial behavior your application uses.

This is one reason I don't treat a green test suite as proof that AI-generated code is safe. A test suite can faithfully confirm the same misunderstanding that produced the code.

Negative testing is a separate topic, and probably worth its own article.

The review question I use

When I review security-sensitive AI-generated code, I don't ask:

“Is there authentication?”

I ask:

“Is authorization enforced for this exact operation on this exact resource?”

That wording matters. It forces the review away from the comforting presence of middleware, current_user, role checks, and authentication helpers.

You have to follow the access path.

For an object lookup, that means tracing from the request down to the data access:

request
  ↓
authenticated user
  ↓
authorization decision
  ↓
specific resource
  ↓
data access

If the chain contains a jump like:

authenticated user
  ↓
Model.find(params[:id])

I want to know exactly what prevents cross-user or cross-tenant access. “Auth is handled elsewhere” isn't an answer until you verify where and how.

A useful AI review prompt

One of the prompts I use in the Security Deep-Dive is deliberately explicit about this failure mode:

Review this code for missing or incorrect authorization checks.

For every resource accessed by ID or other user-controlled identifier:

- identify who is allowed to access it
- verify that authorization is checked for the specific resource
- do not treat authentication alone as authorization
- flag any lookup where an authenticated user could substitute another object's ID

For each finding, include:
- severity
- file and line number
- exploit path
- minimal fix

If authorization depends on middleware, policies, framework defaults, or another layer, state that assumption and mark it as needing verification rather than inventing a vulnerability.

The last sentence is important: security review prompts can become useless if you tell the model to be “aggressive” and it responds by inventing vulnerabilities that depend on imaginary configuration. I want it to challenge the code, but I also want findings grounded in what is actually there.

Try the evil-user version

For security-sensitive changes, I often do a second pass from the opposite direction.

Instead of:

“Review this code for security problems.”

ask:

“Assume you are a malicious but valid user of this system.”

That user already has credentials. That removes the easy answers.

The question becomes:

Can I change an object ID?
Can I access another account's data?
Can I call an action I should not have permission to call?
Can I escalate from one valid permission into another?
Can I combine two individually minor behaviors into something more serious?

A surprising number of access-control mistakes only become obvious once you stop imagining an anonymous attacker and start imagining a perfectly legitimate user who is curious about what happens when they change a parameter.

Where I spend extra review time

I treat this pattern as especially important in changes involving:

user-owned records
accounts and organizations
multi-tenant data
billing
admin functionality
exports
file access
APIs that accept object IDs
background jobs acting on behalf of users
anything involving customer data

Those are the places where “logged in” and “allowed to do this” most obviously diverge.

The rule I keep coming back to

AI-generated security bugs aren't always dramatic. They don't necessarily contain eval() or expose a secret in plaintext. Sometimes the code uses the correct framework, the correct authentication system, the correct ORM, and the correct controller structure. It just misses one relationship.

This user. This operation. This resource.

That is enough.

Authentication answers:

Who are you?

Authorization answers:

Are you allowed to do this to that?

Don't let an AI-generated authentication check convince you that the second question has been answered.

This is one of the checks in Round 2: Security Deep-Dive of The AI Code Review Protocol.

If this way of reviewing code is useful, my free repo contains the free prompts, and the book explains the full four-round method and why each round exists.

The free prompt library and automated review skill are available in the GitHub repository.

If you want the reasoning behind the full four-round process, the guide is available on Selar and Amazon Kindle.

I Reviewed 200+ AI-Generated PRs. Here's the 4-Round Protocol I Use Now.

Stephen Metcalfe — Mon, 15 Jun 2026 12:54:08 +0000

Your teammate used Claude to generate a new API endpoint. The code looks great — clean formatting, proper error handling, even comments. You skim through it, see it follows conventions, CI is green. You approve.

Two weeks later, the endpoint silently drops a decimal place on currency conversions. A financial report is wrong for three days before anyone notices.

This scenario is playing out in hundreds of teams right now. Not because AI generates "bad code" — but because AI-generated code fails in ways human code doesn't, and your existing review process wasn't designed for it.

The Problem With Reviewing AI Code

AI doesn't flag uncertainty. It presents everything with equal confidence. A human developer might write // not sure about the caching here — that nervous comment tells you exactly where to look. AI never writes that comment. It writes // Transform the input to match the expected schema with full confidence, even when the transformation is wrong.

After reviewing hundreds of AI-generated PRs over the past year, I found a pattern. The bugs aren't in formatting. They're in the places a quick glance won't reach:

Off-by-one errors in loops you skimmed
Missing auth checks on new endpoints
Elegant abstractions that create maintenance nightmares
Code that solves the wrong problem perfectly

Generic "review this code" prompts won't catch these. You need a system.

The 4-Round Protocol

I built a review protocol specifically for AI-generated code. Four rounds, each targeting a different failure mode. Total time: ~15 minutes for a typical PR, up to 35 minutes for a large one.

Round	Name	What You're Catching
1	Surface Scan	Logic errors, off-by-one, wrong assumptions
2	Security Deep-Dive	Injection, auth gaps, data leaks
3	Architecture Smell Check	Wrong patterns, tech debt, doesn't fit the system
4	Comparison Pass	Does this match what we actually asked for?

The key insight: each round uses a separate AI prompt that forces a different lens on the same code. You're not asking the AI to "review this code" four times — you're asking four different, targeted questions.

Let me show you the two rounds that catch the most issues.

Round 1: The Surface Scan

This is the high-probability round. Most AI bugs live here — logic errors, wrong assumptions, off-by-one bugs. The code looks correct. It's subtly wrong in exactly the ways a quick glance won't catch.

Here's the prompt I use:

Review this code for logic errors only. Do NOT suggest style improvements,
documentation, or refactoring. I want you to find:

1. Off-by-one errors, wrong comparisons, or inverted logic
2. Wrong default values or assumptions about data shape
3. Missing edge case handling (null, empty, zero, max values)
4. Race conditions or non-atomic operations on shared state

For each issue found, state:
- The exact line number
- Why it's wrong
- What the correct behavior should be

If you find zero issues, explain why each edge case IS handled,
not just say "looks good."

[Paste the PR description or requirements if available]

The critical instruction is that last line: "If you find zero issues, explain why each edge case IS handled." Without this, the AI will happily say "looks good" and move on. Forcing it to justify the all-clear catches things a simple yes/no never will.

The trap: AI-generated tests will pass. AI knows what the code does, so it writes tests that confirm the code's behavior — including its bugs. Perfect test coverage means nothing if the tests are testing the wrong thing.

Round 2: The Security Deep-Dive

This is the scary one. AI models are trained on massive amounts of public code, including code with security vulnerabilities. They don't understand security — they understand patterns. If the most common Stack Overflow solution uses eval() or concatenates SQL strings, the AI will reproduce that pattern with full confidence.

The most common AI security failures: SQL injection, insecure deserialization (pickle, Marshal, YAML.load), BOLA/IDOR (authenticated but accessing someone else's resource), mass assignment, and SSRF.

Here's the prompt that catches what your brain won't think to check:

You are a malicious actor with valid API credentials who want to exploit
this code. Walk through every possible thing you could try:

- Can you access data you shouldn't be able to?
- Can you escalate privileges?
- Can you cause the system to leak internal information?
- Can you trigger unexpected behavior with edge inputs?
- Can you cause the system to consume excessive resources?

Think step by step. List at least 5 distinct attack vectors. If you can't
find 5, you're not thinking creatively enough.

After listing individual vectors, describe at least 2 attack chains where
you combine multiple steps to achieve something none of the individual
vectors accomplish alone.

Pro tip: use a different AI model for this round than the one that generated the code. If Claude wrote the code, use another model (if possible, from another provider) to review it. Different training data means different blind spots. This single change catches vulnerabilities that using the same model consistently misses.

The trap: Auth checks that look right but aren't. AI will write if current_user.present? — the user is authenticated, but the code doesn't check if they're authorized for that specific resource. The check looks secure but isn't.

The Two Rules That Make This Work

1. Clear context between rounds. Don't run all 4 rounds in the same chat thread. Start a fresh conversation for each round. If you run Round 2 in the same context as Round 1, the AI already "knows" what it told you in Round 1 and will unconsciously align its analysis. Fresh context forces independent analysis. Costs 30 seconds, worth every one of them.

2. Run all reviews first, fix once. The naive approach is to fix issues one at a time — fix the logic bug, review, fix the security hole, review. This creates whack-a-mole: fixing the architecture can introduce a new security hole. Instead: run all 4 rounds, collect every issue, send the complete list to the AI in one shot, then re-run all rounds on the result.

When to Skip a Round

Not every PR needs all 4 rounds:

Scenario	Rounds to Run
Comment or docs change	None
Variable rename	None
Typo fix	1 & 4
New API endpoint	1, 2 & 4 (skip 3 if it follows existing patterns)
New feature, new patterns	All 4
Auth or payment change	All 4 — extra time on round 2
AI-generated bugfix	All 4 — the fix might work but introduce new bugs

General principle: if AI generated the code, lean toward running more rounds. That's the whole point.

The Round Everyone Skips

Round 4 — the Comparison Pass — is the most commonly skipped and the most dangerous to skip. It asks one question: does this code actually solve the problem we asked for?

AI is excellent at solving the problem you typed, not the problem you meant. It takes your words literally. The most common failure: AI solves the first 80% of a ticket perfectly and quietly ignores the last 20% because it "didn't seem important." The code is perfect — for the wrong thing.

If you have a ticket or issue, paste it in and make the AI verify each acceptance criterion against the code. You'll be surprised what's missing.

Making It Stick

Here's how to adopt this without overwhelming yourself:

Day 1-2: Run only Round 1 on all your PRs. Get comfortable with the prompts.
Day 3-4: Add Round 2. You'll likely find something within the first few PRs.
Day 5-6: Add Rounds 3 and 4.
Day 7: Reflect. What failure patterns did you see most? Which round caught the most issues?

The best review process is the one that evolves. When you find a pattern this protocol doesn't catch, add your own round. When a prompt stops finding bugs, retire it.

Where This Comes From

I've been using versions of this protocol for over a year. It has saved me from shipping bugs that I would have approved on a first pass. Not every time, but often enough that running it is automatic now.

I wrote the full protocol — all 4 rounds, 12 copy-paste prompts, the "traps to watch for" in each round, a printable checklist, and the review loop workflow — into a guide. It's called The AI Code Review Protocol and it's on Selar for $8.99.

If you want the complete version with Rounds 3 and 4, the architecture smell checklist, the PII audit prompt, and the automation approaches — that's there. If this post was useful, the guide goes deeper.

If you've built your own review process for AI code, I'd genuinely like to hear what works for you. I'm @raithlin on X, or drop a comment below.