DEV Community

Cover image for Why my Solidity test loop went from 56 seconds to under one
borodutch.eth 💎🤲
borodutch.eth 💎🤲

Posted on Edited on Originally published at blog.borodutch.com

Why my Solidity test loop went from 56 seconds to under one

The contracts for a game I'm building have 303 tests, compiled with via-IR, and the biggest test file is 7,500 lines long. Editing one test and rerunning the suite took 56 seconds with Foundry and 61 with Hardhat 3. With plain, the toolchain I wrote, the same edit and rerun takes 0.99 seconds, on the same project, with the same foundry.toml and nothing changed in the code. Solc is the same too: what gets deployed is byte for byte what solc produces. The speed comes from not doing work twice, and this post is about where that work hides.

All numbers below are medians from hyperfine on an M1 Max, measured on October 17, 2026, with forge 1.7.1, Hardhat 3.20.0 and the solc each project pins. Foundry / Hardhat / plain:

The game (via-IR) Uniswap v4-core (via-IR) solmate
Edit a test, run the tests 56 s / 61 s / 0.99 s 24 s / 26 s / 1.8 s 4.6 s / 4.2 s / 0.07 s
Edit a contract function, run the tests 103 s / 109 s / 2.1 s 130 s / 132 s / 4.2 s 6.0 s / 5.7 s / 1.5 s
Build from scratch 116 s / 115 s / 15 s 141 s / 144 s / 12 s 5.6 s / 6.8 s / 1.1 s

Every test passes under plain: 303 in the game, 598 in v4-core and 568 in solmate.

Tests compile your contracts again

This one surprised me the most. When a test does new Vault(args), solc has to put Vault's whole bytecode inside the test contract, so it compiles and optimizes Vault again, once per test file. With via-IR, that repeated work is most of the build.

plain compiles test files with each new X(...) replaced by a deploy of X's already compiled artifact, with the same init code, deployer, nonce and address. The constructor arguments are still type-checked against X, so a wrong argument is still a compile error. A side effect: since tests no longer contain your contracts' bytecode, editing a function in src/ doesn't recompile the tests at all. A test file is cached by its own source plus the interfaces of what it imports, with function bodies left out.

A big test file doesn't compile as one unit

A 7,500-line test file is one huge job for solc. plain splits it into several units that compile at the same time, each holding only its share of the test functions; the rest are cut out of the source, so solc doesn't even type-check them. Each unit remembers which tests it holds. When you edit test bodies, the untouched units stay cached and the edited or new tests compile into one small "hot" unit. That's why editing a test costs about a second instead of a minute.

Unchanged tests don't run again

A suite's results are reused while its bytecode, the settings and everything it read (deployed artifacts, files, environment variables) stay the same, and cached suites print (cached). Tests are deterministic, with a fixed fuzz seed and no wall clock, so this is safe. Anything that isn't deterministic always reruns: suites that call ffi, writeFile, unixTime or setEnv, and every fork test. --rerun runs everything anyway.

The rest is scheduling and caching

  • Parallel solc. Compile jobs go on a work queue, biggest first, sized from the last build's timings. Foundry and Hardhat run one solc process at a time.
  • Parallel tests. Suites, tests and individual fuzz and invariant runs execute in parallel on an in-process revm, each starting from a copy-on-write view of the post-setUp state.
  • A shared cache. Results are keyed by content hash and kept in ~/.plain/cache, so a new git worktree or a fresh clone of already built sources runs its tests in 0.06 seconds instead of doing a cold build. That matters a lot when AI agents work in their own worktrees.
  • Only what's needed. plain test Vault compiles only the test files that filter can match.

In CI

I also ran Uniswap v4-core on GitHub Actions' 2-core runners, taking the median of repeated runs. With the build cache restored and one line changed in PoolManager.swap, forge build took 3 minutes 1 second at 4.3 GB of memory, and plain build took 2.6 seconds at 0.1 GB. A cold build from nothing is a smaller win on two cores: 3 minutes 57 seconds for Foundry against 1 minute 17 seconds for plain, at 5.0 GB and 0.7 GB of memory. Parallel solc needs cores to shine.

What it runs

plain runs Foundry-style Solidity tests with fuzzing, invariants and the common cheatcodes, plus TypeScript tests against a fresh chain per file. It reads foundry.toml, so existing projects work as they are, and the README lists the supported cheatcodes if your suite leans on unusual ones. Tell me how it goes on your project.

Try it on your project

curl -fsSL https://getplain.org/install | sh
cd your-foundry-project
plain test --watch
Enter fullscreen mode Exit fullscreen mode

That compiles and runs every test, then again on every save. It's also on Homebrew, npm (npx plain-evm test), apt, dnf, the AUR, mise, Nix and crates.io. The docs cover testing, deploying with verification on Sourcify, Etherscan and Blockscout, and upgrades, and the source and the full benchmark notes are on GitHub.

Top comments (4)

Collapse
 
bloqarl profile image
Carlos | Zealynx •

The new X(...) part was news to me, it explains why via-IR suites feel so slow. One thing I'd watch with the result cache: with a fixed seed, a fuzz or invariant test that passed once doesn't see a new input again until the code changes. Forge leaves fuzz.seed unset by default, so every CI run is a fresh draw, and over a few weeks that's a lot more inputs than the configured runs. Is there a way to rotate the seed in CI, or does --rerun pick a new one?

Collapse
 
backmeupplz profile image
borodutch.eth 💎🤲 •

Good catch, and you're right that the fixed seed trades fresh inputs for a cache you can trust. plain's default seed is fixed, and --rerun runs everything again with that same seed, so it won't give you new inputs. To rotate it in CI, pass one, for example plain test --seed $GITHUB_RUN_NUMBER (--fuzz-seed works too). The seed is part of the cache key, so a new seed reruns the suites instead of printing (cached). I keep the fixed seed locally for the fast edit loop and let CI draw a new one every run, which gets you Forge's fresh-draw behavior where it matters.

Collapse
 
bloqarl profile image
Carlos | Zealynx •

$GITHUB_RUN_NUMBER is a good pick for another reason: it stays the same when you hit re-run on a failed job, only GITHUB_RUN_ATTEMPT goes up. So a red fuzz run replays the same inputs on re-run instead of drawing new ones and maybe going green, which is how a real counterexample gets waved off as flaky.

Thread Thread
 
backmeupplz profile image
borodutch.eth 💎🤲 •

Yep! I've added the flag in the latest version that people can use, setting it to $GITHUB_RUN_NUMBER :) thank you!