How a CI Performance Budget Keeps a 6 KB Framework From Growing (Inside AngeLite, Part 4)

#webdev #testing #performance #javascript

Every framework starts fast and gets slower. Features accumulate, each justified on its own, and one day the "lightweight" library isn't. AngeLite's defence is a rule: no feature lands if it makes the core slower, bigger or hungrier for memory, and the rule is enforced by CI, not by good intentions.

This post explains the four layers of that enforcement and why they're arranged from cheapest and exact to expensive and noisy.

The problem with measuring time in CI

The obvious approach is "run the benchmark on every PR and fail if it got slower". It doesn't work. On a shared CI runner, time varies 10-20% from run to run, so a threshold tight enough to catch regressions is loose enough to flake constantly.

What is exact: how much work the code does (DOM calls, effect runs) and how many bytes it ships. So the budget leans on exact measurements first and uses noisy ones only as a backstop.

The four layers

Layer	What it measures	Where it runs	Noise
1. Size	gzip bytes of every bundle, and of what a user really downloads (the table app built with esbuild)	CI, every PR	none
2. Work	DOM calls and effect runs per scenario (1,000 rows) against a stored budget	CI (`npm test`), every PR	none: deterministic
3. Relative time	AngeLite vs Preact in the same run; ratio compared with a baseline	on demand, weekly, and PRs touching the core; doesn't block	medium
4. Full benchmark	five frameworks, 12 measurements, memory and startup	by hand before publishing	high

flowchart TB
    PR[Pull request] --> L1["Layer 1: bundle size (exact)"]
    PR --> L2["Layer 2: work budget (exact)"]
    L1 --> Gate{"Within budget?"}
    L2 --> Gate
    Gate -- no --> Fail["CI fails and names the figure"]
    Gate -- yes --> Merge[Mergeable]
    PR -. "touches core" .-> L3["Layer 3: relative time vs Preact (noisy, non-blocking)"]
    Release[Before publishing] --> L4["Layer 4: full 5-framework benchmark (by hand)"]

Layer 2 in detail: count the work, not the time

tests/perf-budget.test.mjs mounts the table app with every optional module loaded (portal, @defer, animations...) and counts, per scenario, calls to createElement, cloneNode, appendChild, insertBefore, remove, setAttribute, classList.toggle, textContent =, addEventListener and so on, plus Effect.run executions. It also asserts that the in-browser compiler and the precompiled template do exactly the same work. Today, for 1,000 rows:

Scenario	DOM calls	Effects
create 1,000	5,002	1,001
replace 1,000	5,004	1,001
update every 10th	100	101
select a row	1	1,000
swap 2 rows	2	1
remove a row	1	1
append 1,000	5,000	1,001
clear	4	1

If a change raises any figure, the test fails and says which one. If it's intentional, you explain it in the PR and run UPDATE_PERF=1 npm test, which rewrites the budget in the same commit, so the change is visible in review. And if a figure drops, the test says so too, so the improvement gets locked in.

This catches things time can't: a feature that adds one DOM call per row won't move a noisy millisecond number, but it changes a count of 5,002 to 6,002 and fails immediately.

Layer 3: relative time, because absolute time lies

Comparing AngeLite against Preact within the same run cancels most machine noise. A single operation fails if its ratio gets worse by more than 35%, or the geometric mean of all operations by more than 10%. It doesn't block merges because it's still statistical, but it catches what the exact layers miss, such as a new property access in a hot loop.

Design rules that keep the budget achievable

Optional module by default. A feature not every app needs goes in its own file and hooks in through the ops registry or a module imported only when used. Apps that don't import it don't pay in bytes or in time.
Don't touch the hot paths without measuring. They are instantiate, runView, ForState.reconcile and removeView. A new property on every row gets multiplied by thousands.
Pay per component or template, not per row. Compile-time work in the AOT is nearly free; per-row work isn't.
Mind object shapes. Adding fields to View, to the context or to elements changes their shape and can slow everything that touches them. Optional fields are created only when used (DI providers do this: the map is created on the first inject).
Dev warnings don't live in the core. enableDevWarnings() wraps internals from the outside; the production build includes none of it.
Every byte of the core is debated. Size budgets live in scripts and only go up in a commit that explains why.

What a budget buys you: an audit trail

Because budgets are data in the repo, the cost of every core feature is public. Before the "wave 1" feature work the precompiled table app was 5,968 bytes gzip; today it's 6,223 (+255 B, +4%). Post 5 breaks down where those bytes went.

Pending improvements, with data

The budget also acts as a to-do list, ordered by what the numbers already show:

Selecting a row runs 1,000 effects (every row reads selected() and is re-evaluated to change a class on two rows). About 1.3 ms and tied with Preact and Angular, but it's the best candidate for per-binding effects.
create1k does 5 DOM calls per row, two of which (a text write of the initial value and a classList.toggle to false) can be skipped.
remove/swap hover around a tie with Preact. A Chromium trace profile is still missing to separate noise from real cost before touching anything.

Takeaways

Measure exact things (bytes, DOM calls, effect runs) on every PR; use noisy time only as a ratio and only as a backstop.
Make budgets data in the repo, with an explicit update command, so every change is reviewed.
A performance rule you can't break by accident is worth ten you remember to follow.

Next: the module system that makes "optional" real, and what each core feature costs.

DEV Community