DEV Community

ohmygod
ohmygod

Posted on

Mutation Testing for Smart Contracts: How Slither-Mutate + Foundry Reveals the Tests You Forgot to Write

#security #solidity #defi #ethereum

Your test suite passes. Coverage is 95%. The audit report is clean. You deploy. Three weeks later, someone drains $4.2 million because a boundary condition your tests never actually checked.

This isn't hypothetical. Over 70% of exploited contracts in 2025-2026 had passed professional audits. The problem isn't that teams don't test — it's that they don't test whether their tests are actually catching bugs.

Enter mutation testing: the practice of deliberately injecting synthetic bugs into your code, then checking if your test suite catches them. If a "mutant" survives — meaning your tests still pass with a bug present — you've found a gap that a real attacker could exploit.

In this guide, we'll walk through setting up slither-mutate with Foundry to systematically find the holes in your DeFi test suite, with real patterns from recent exploits that mutation testing would have flagged.

Why Line Coverage Lies

Consider this simplified lending function:

function withdraw(uint256 shares) external {
    uint256 assets = convertToAssets(shares);
    require(assets <= totalAssets(), "Insufficient liquidity");
    _burn(msg.sender, shares);
    IERC20(asset).transfer(msg.sender, assets);
}
Enter fullscreen mode Exit fullscreen mode

Your test calls withdraw(100) with sufficient balance, and it passes. Coverage tool marks the function as 100% covered. But did you test:

  • withdraw(0) — the zero-share edge case?
  • withdraw(type(uint256).max) — overflow in convertToAssets?
  • Withdrawal when totalAssets() has been manipulated via donation?
  • The ordering of _burn before transfer (reentrancy)?

Line coverage says "tested." Mutation testing says "prove it."

Setting Up the Pipeline

Prerequisites 

# Install Foundry
curl -L https://foundry.paradigm.xyz | bash
foundryup

# Install Slither (requires Python 3.10+)
pip install slither-analyzer
Enter fullscreen mode Exit fullscreen mode

Running Your First Mutation Campaign

Navigate to your Foundry project root and run:

slither-mutate src/ --test-cmd='forge test' --output-dir=mutation_campaign
Enter fullscreen mode Exit fullscreen mode

This does three things:

  1. Parses your Solidity source and generates mutants (small code changes)
  2. Runs forge test against each mutant
  3. Reports which mutants survived (your tests didn't catch the injected bug)

For targeted analysis on a specific contract:

slither-mutate src/core/LendingPool.sol \
  --test-cmd='forge test --match-contract="LendingPoolTest"' \
  --verbose \
  --output-dir=mutation_campaign
Enter fullscreen mode Exit fullscreen mode

The Mutant Zoo: What Gets Changed

Slither-mutate applies several categories of mutations. Understanding them helps you interpret results:

1. Arithmetic Operator Replacement 

// Original
uint256 fee = amount * feeRate / 10000;
// Mutant: * → +
uint256 fee = amount + feeRate / 10000;
Enter fullscreen mode Exit fullscreen mode

Why it matters: The March 2026 Resolv exploit succeeded partly because minting calculations weren't properly bounded. If your fee/rate calculations have surviving arithmetic mutants, an attacker might find a way to manipulate the math.

2. Relational Operator Replacement 

// Original
require(collateralRatio >= MIN_RATIO, "Undercollateralized");
// Mutant: >= → >
require(collateralRatio > MIN_RATIO, "Undercollateralized");
Enter fullscreen mode Exit fullscreen mode

Why it matters: Off-by-one errors in boundary checks are a classic DeFi vulnerability. The Venus Protocol donation attack exploited exactly this kind of boundary weakness in supply cap logic.

3. Conditional Boundary Mutations 

// Original
if (block.timestamp > deadline) revert Expired();
// Mutant: > → >=
if (block.timestamp >= deadline) revert Expired();
Enter fullscreen mode Exit fullscreen mode

Why it matters: These boundary mutations reveal whether your tests actually check the exact boundary conditions, not just the happy path.

4. Statement Deletion 

// Original
_updateAccrued(user);
uint256 debt = borrowBalance[user];
// Mutant: statement deleted
uint256 debt = borrowBalance[user];
Enter fullscreen mode Exit fullscreen mode

Why it matters: The Planet Finance exploit (March 2026) succeeded because the protocol mistakenly treated stored borrow balance increases as accrued interest. A mutation that deletes the accrual update would test whether your suite catches stale state.

Real-World Pattern: Catching the DBXen Identity Bug

The March 2026 DBXen exploit ($149K loss) stemmed from inconsistency between _msgSender() and msg.sender. Here's how mutation testing flags this class of bug:

// Your contract uses ERC2771 meta-transactions
function stake(uint256 amount) external {
    address user = _msgSender(); // Correct: supports meta-tx
    _stake(user, amount);
}

function unstake(uint256 amount) external {
    address user = msg.sender; // BUG: doesn't support meta-tx
    _unstake(user, amount);
}
Enter fullscreen mode Exit fullscreen mode

Slither-mutate would generate a mutant replacing _msgSender() with msg.sender (or vice versa). If your test suite doesn't test meta-transaction paths, both the original and mutant pass — the mutant survives, flagging the inconsistency.

The fix: Add a test that calls stake and unstake via a trusted forwarder and verifies the correct user is credited/debited.

Interpreting Results: A Triage Framework

After running a campaign, you'll see output like:

[SBR] Line 142: '>=' ==> '>' --> UNCAUGHT
[AOR] Line 87: '*' ==> '+' --> CAUGHT  
[CR]  Line 203: 'require(x > 0)' ==> '' --> UNCAUGHT
[SDL] Line 156: '_updateRewards(user)' ==> '' --> UNCAUGHT
Enter fullscreen mode Exit fullscreen mode

Triage by severity:

Priority Mutant Type Risk
🔴 Critical Statement deletion (SDL) of access control, auth checks Direct exploit path
🔴 Critical Conditional removal (CR) of require/revert Validation bypass
🟠 High Arithmetic operator (AOR) in financial calculations Value manipulation
🟡 Medium Relational boundary (SBR) changes Edge case exploits
🟢 Low Cosmetic mutations (string changes, event params) Usually non-exploitable

Focus on the red and orange first. Every surviving critical mutation is a potential exploit path.

Advanced: Targeted Mutation Campaigns for DeFi

Audit-Critical Functions Only

Don't mutate your entire codebase — focus on the money:

# Mutate only the vault and lending contracts
slither-mutate src/core/Vault.sol src/core/LendingPool.sol \
  --test-cmd='forge test -vvv' \
  --timeout 120 \
  --verbose
Enter fullscreen mode Exit fullscreen mode

Combining with Foundry Invariant Tests

Mutation testing is most powerful when paired with invariant tests:

function invariant_totalAssetsBacksSharesTotalSupply() public {
    assertGe(
        vault.totalAssets(),
        vault.convertToAssets(vault.totalSupply())
    );
}
Enter fullscreen mode Exit fullscreen mode

Invariant tests catch a broader class of mutations because they assert properties rather than specific input/output pairs. Run your mutation campaign with invariant tests included:

slither-mutate src/core/Vault.sol \
  --test-cmd='forge test --match-contract="VaultInvariantTest"'
Enter fullscreen mode Exit fullscreen mode

Custom Mutators for DeFi-Specific Patterns

List available mutators:

slither-mutate --list-mutators
Enter fullscreen mode Exit fullscreen mode

For DeFi audits, prioritize these mutators:

  • AOR (Arithmetic Operator Replacement) — catches fee/rate calculation bugs
  • SBR (Solidity Boolean Replacement) — catches access control bypasses
  • CR (Condition Removal) — catches missing validation
  • SDL (Statement Deletion) — catches skipped state updates 
slither-mutate src/ \
  --test-cmd='forge test' \
  --mutators-to-run AOR,SBR,CR,SDL
Enter fullscreen mode Exit fullscreen mode

Building It Into CI/CD

Add mutation testing as a weekly CI job (it's too slow for every commit):

# .github/workflows/mutation-test.yml
name: Weekly Mutation Testing
on:
  schedule:
    - cron: '0 6 * * 1'  # Every Monday at 6 AM UTC
  workflow_dispatch:

jobs:
  mutation-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: foundry-rs/foundry-toolchain@v1
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install slither-analyzer
      - run: forge build
      - run: |
          slither-mutate src/core/ \
            --test-cmd='forge test' \
            --output-dir=mutation_campaign \
            --verbose
      - uses: actions/upload-artifact@v4
        with:
          name: mutation-results
          path: mutation_campaign/
Enter fullscreen mode Exit fullscreen mode

The Mutation Testing Mindset

Mutation testing inverts your perspective. Instead of asking "do my tests pass?" you ask "would my tests fail if this code were wrong?"

Every surviving mutant is a question: If an attacker found a way to make the code behave like this mutant, would anyone notice?

For the DeFi protocols that lost $1.66 million in a single week in March 2026 — from oracle misconfigurations to flawed liquidation logic to identity confusion bugs — the answer was no. Their tests passed. Their coverage was high. But their tests weren't actually testing the things that broke.

Mutation testing won't catch everything. It won't find economic exploits that require multi-step flash loan choreography. But it will systematically reveal the assertions you forgot to write, the boundaries you forgot to check, and the state updates you forgot to verify.

In smart contract security, the bugs that drain millions are rarely exotic. They're the simple ones hiding in the gaps between the tests you wrote and the tests you thought you wrote.

This article is part of a series on practical DeFi security tooling. Previous entries covered Solana CPI vulnerability patterns, runtime security monitoring, and Foundry invariant testing.

Top comments (0)