DEV Community

137Foundry
137Foundry

Posted on

How to Write a Permission Test Matrix That Actually Catches Regressions

Most permission bugs that reach production don't fail loudly. Over-permissive bugs leak data quietly until someone notices something they shouldn't have seen. Under-permissive bugs generate support tickets that rarely get traced back to the actual missing check. A test suite that only covers the happy path catches neither. Here's how to build one that does.

Step 1: List Every Role and Every Protected Action Separately

Before writing a single test, make two flat lists: every role in your system, and every action your can() function gates. Don't try to write the tests yet. This step alone often surfaces gaps, like an action nobody remembered to add a role for, or a role that has no tests referencing it at all.

Keep the lists in a plain text file or spreadsheet next to your test suite, not buried in code comments. It should be trivially easy for a reviewer to check "is there a test for this role against this action" without reading test implementations.

Step 2: Build the Matrix, Not a List

The mistake most teams make is writing tests as a list: "admin can delete," "editor can edit," "viewer can view." That covers the cases someone thought to write down, and misses everything else. A matrix tests every role against every action explicitly, asserting both the allow and the deny case.

describe.each(ROLES)('permission matrix for %s', (role) => {
  test.each(ACTIONS)('action: %s', (action) => {
    const expected = EXPECTED_MATRIX[role][action];
    expect(can({ role }, action)).toBe(expected);
  });
});
Enter fullscreen mode Exit fullscreen mode

The EXPECTED_MATRIX constant becomes your source of truth, readable at a glance, and it's the artifact a reviewer checks against the product requirement rather than reading through dozens of individual assertions.

Step 3: Make the Deny Case a First-Class Assertion

It's tempting to only test that a role can do what it's supposed to. The denial case is exactly what regresses silently when someone adds a new permission and forgets to gate a new endpoint behind it. If your matrix only asserts allows, a new action added without a corresponding deny assertion for every other role will pass every existing test while being wide open by default.

Assert both directions for every cell in the matrix, even when it feels redundant. The redundant-feeling assertions are the ones that catch the bug eighteen months from now when nobody remembers the original design intent.

Step 4: Treat a Missing Test as a Blocked PR, Not a Follow-Up Ticket

The cost of writing a permission test at review time is minutes. The cost of discovering a missing one is usually a support escalation and an uncomfortable conversation about what a customer's contractor could see that they shouldn't have. Any new mutating endpoint without an entry in the test matrix should block the pull request, the same way a missing migration would.

This is a policy decision, not just a technical one, and it's worth stating explicitly in your contribution guidelines rather than leaving it to reviewer memory. OWASP's testing guidance treats access control testing the same way, as a required gate rather than an optional nice-to-have.

Step 5: Test Scoped Permissions With Real Resource Boundaries

If your system scopes permissions to teams, tenants, or specific resources, the matrix needs a second dimension beyond role and action: does this permission check correctly fail when the resource belongs to a different scope than the user's? A role that can approve invoices for their own team but not another team's is a different assertion than the basic role/action pair, and it's the specific case that tends to regress when caching or scope-resolution logic changes.

Build fixtures with at least two of every scoped entity (two teams, two tenants) specifically so tests can assert the cross-scope denial explicitly, rather than only testing within a single scope where a missing boundary check would pass silently.

Consider Property-Based Testing for Large Matrices

Once a permission matrix grows past a handful of roles and actions, hand-writing every cell becomes tedious and error-prone in its own right. Property-based testing tools like Hypothesis can generate combinations of roles, actions, and scopes automatically and assert a general property, such as "a user never has an action their role doesn't include," rather than requiring every cell to be listed by hand.

This doesn't replace the explicit matrix for your core roles, which is still the clearest documentation of intended behavior. It's a good complement for catching combinations nobody thought to write down explicitly, especially once scoped permissions add a third dimension that makes manual enumeration genuinely tedious.

If You're Layering in Policy-Based Rules

Teams that eventually add conditional or attribute-based rules on top of role-based access control, using something like Open Policy Agent or Casbin, still need the same matrix discipline, just extended to cover the conditions rather than only the base role/action pairs. A policy engine doesn't remove the need for explicit allow and deny test cases. It just means the matrix now needs entries for "role X with attribute Y should be denied," which is easy to forget if the test suite was built before those rules existed.

Whichever model you're testing, the underlying principle stays the same: every combination that can grant or deny access needs an explicit, automated assertion, because the ones nobody thought to write down are exactly the ones that regress silently.

Keeping the Matrix Current as the Product Changes

A permission test matrix is only useful while it stays accurate, and products change fast enough that an unmaintained matrix drifts out of date within a few months. Make updating the matrix a required step whenever a new role or action is introduced, the same way updating a database migration is a required step when the schema changes, not an optional cleanup task someone gets to eventually.

One practical habit: have the matrix file itself fail a lint check if a new permission string appears in the codebase without a corresponding entry, so the gap surfaces at PR time rather than being caught later during an unrelated bug investigation. This turns "did anyone remember to add a test" into an automated question instead of a hope, which is exactly the kind of guarantee permission logic benefits from most.

Where This Fits Into the Bigger Picture

A test matrix is only as good as the permission model it's testing. If roles and actions are still scattered across ad-hoc boolean flags rather than a real permissions table, there's no clean place to enumerate the matrix from in the first place. Our guide on building a role-based permissions system covers the data model this testing approach assumes, including how scoping and caching interact with the checks you're testing here.

137Foundry has built this exact matrix pattern into CI for several client codebases, and the return on investment tends to show up the first time someone adds a feature under deadline pressure and the test suite catches the missing deny case before it ships.

That moment, a red test blocking a merge instead of a support ticket three weeks later, is the entire argument for building the matrix in the first place. It's a small amount of upfront structure that turns a category of bug that used to be nearly invisible into one CI catches automatically, every time, without relying on anyone remembering to check by hand.

Top comments (0)