LLVM #12 — Testing my Custom Pass on a Billion Additions. It was INSANE!

I've spent the last several posts building a pass that finds loops and attaches a "vectorise this" hint to them. Today I actually tested it on a 1000×1000 matrix addition, run 1000 times. That's a billion element-wise floating point additions in the hot loop. And I kid you not... I was NOT prepared for the result.

1134ms without the pass. 271ms with it.

That's nearly 4x faster. Same source code, same machine, same input. The only difference was one piece of LLVM metadata that changed how the optimiser was allowed to transform the loop.

What I built: Benchmark Experiment History

---

What I did:

1) Writing a 2D matrix-addition:

• I'd been testing this on a flat 1D array. And the results were pretty nice. From 394 ms without the pass to 324 ms with it. But this time, I wanted something that actually deserves to be called matrix addition — two 1000×1000 grids of floats, added element-by-element:

#include <iostream>
#include <chrono>
using namespace std;

const int N = 1000;

void matrix_add(float A[N][N], float B[N][N], float C[N][N]) {
    for (int i = 0; i < N; i++) {
        for (int j = 0; j < N; j++) {
            C[i][j] = A[i][j] + B[i][j];
        }
    }
}

int main() {
    static float A[N][N], B[N][N], C[N][N];

    for (int i = 0; i < N; i++)
        for (int j = 0; j < N; j++) {
            A[i][j] = i + j;
            B[i][j] = i * j;
        }

    auto start = chrono::high_resolution_clock::now();
    for (int t = 0; t < 1000; t++)  // run 1000 times to make timing meaningful
        matrix_add(A, B, C);
    auto end = chrono::high_resolution_clock::now();

    cout << chrono::duration_cast<chrono::milliseconds>(end - start).count() << " ms\n";
    cout << C[500][500] << "\n";  // this is just to test it with -o3, so the compiler doesn't abandon the computation.

    return 0;
}

• There were no function calls or external dependencies, like we had last time with printf. A billion additions were enough to make the timing difference real and impossible to dismiss.

2) Compiling to raw IR:

clang++ -S -emit-llvm -O0 -Xclang -disable-O0-optnone -fno-discard-value-names matxAdd2D.cpp -o matxAdd2D_raw.ll

• We compile to LLVM IR with no optimisations. The -disable-O0-optnone flag is essential because without it, -O0 stamps every function with optnone, which then blocks all subsequent opt passes from running.

3) Canonicalising the IR to give the loops structure:

opt -passes='mem2reg,loop-simplify,loop-rotate' matxAdd2D_raw.ll -S -o matxAdd2D_canonical.ll

• We clean up the loop shape, promoting stack variables to SSA registers, giving each loop a single clean preheader and latch, and rotating it into the form the vectoriser expects.

4) Generating a binary out of our pass and the vectoriser:

opt -load-pass-plugin=./build/libMyPass.so -passes="my-pass,loop-vectorize" -S matxAdd2D_canonical.ll -o matxAdd2D_vectorized.ll
clang++ -O0 matxAdd2D_vectorized.ll -o runWithPass2D

• Our pass runs first, finding the inner loop and attaching !llvm.loop.vectorize.enable = true to its latch branch.
• Then LLVM's built-in vectoriser runs immediately after, reads that hint, and rewrites the loop to use SIMD.
• The vectorised IR gets compiled to a binary.

5) Generating the binary without our pass:

clang++ -O0 matxAdd2D_canonical.ll -o runWithoutPass2D

• Here, we compile the canonical IR directly without invoking the vectoriser, so the loop remains scalar.
• Each floating-point instruction operates on one scalar value instead of a vector containing multiple values.

6) The result:

./runWithPass2D    → 271 ms
./runWithoutPass2D → 1134 ms

• Four times faster! On a billion floating point additions, the difference between scalar and vectorised is 890 milliseconds of wall-clock time.
• As a sanity check, I ran it with -O3, which clocks in at 140ms.
• It is obviously much faster because it runs 50+ passes including everything we did and more. But the point was never to beat -O3, and was instead to understand one piece of what's inside it.

clang++ -O3 matxAdd2D.cpp -o runO3
./runO3 → 140 ms

---

Understanding the Assembly

• To understand what's actually happening, I dumped both to assembly and grepped for floating point and vector instructions:

1) Without the pass:

grep -E "fadd|fmul|v[0-9]|\.4s|\.2s" matxAdd2D_plain.s | grep -v "cfi"

fadd s0, s0, s1

• Here, one instruction repeats a billion times.
• Both s0 and s1 hold exactly one float.
• We just add them, store the result, and move to the next element, one at a time.

2) With pass:

grep -E "fadd|fmul|v[0-9]|\.4s|\.2s" matxAdd2D_vectorized.s | grep -v "cfi"

fadd.4s v3, v3, v7
fadd.4s v2, v2, v6
fadd.4s v1, v1, v5
fadd.4s v0, v0, v4
fadd.4s v0, v0, v1
fadd s0, s0, s1

• Here, v0–v7 are 128-bit NEON registers, each holding 4 floats (32-bit) packed side by side.
  • NEON registers are special vector registers in ARM CPUs that let the processor operate on multiple pieces of data at the same time, hence enabling SIMD.
• .4s means "perform this on 4 singles at once."
• The vectoriser also unrolled the loop (four chunks of 4 per iteration, so 16 elements per pass through the loop):

v0 = [ C[j+0]  | C[j+1]  | C[j+2]  | C[j+3]  ]
v1 = [ C[j+4]  | C[j+5]  | C[j+6]  | C[j+7]  ]
v2 = [ C[j+8]  | C[j+9]  | C[j+10] | C[j+11] ]
v3 = [ C[j+12] | C[j+13] | C[j+14] | C[j+15] ]

• All four additions happen in parallel. With the pass, we have about ~62.5 million iterations of 16 elements each.
• The fadd s0, s0, s1 at the end is the scalar tail, handling leftover elements when the count isn't a clean multiple of 16.
• Each row has 1000 elements: 62 full NEON chunks of 16, plus 8 leftovers. The scalar tail handles those 8.

3) The other instructions in the vectorised assembly:

movi.4s v0, #4
dup.4s  v1, w8
add.4s  v4, v5, v1
scvtf.4s v17, v17
mul.4s  v5, v2, v5

• These aren't from matrix_add. They're from the initialisation loops in main that fill A and B with values.
• Our pass practically ran on every function it visited, so the vectoriser rewrote the setup code too.
• scvtf converts integers to floats 4 at a time.
• mul.4s computes i * j for four index pairs simultaneously.
• The hint helped perform vectorisation everywhere, not just in matrix_add.

4) Another remark in the pass output:

remark: <unknown>:0:0: loop not vectorized: call instruction cannot be vectorized

• That's the outer for (int t = 0; t < 1000; t++) timing loop in main, which calls matrix_add. This is, from the vectoriser's perspective, an external function call.
• Like we saw last time, a function call cannot be split across SIMD lanes, so the vectoriser declined that one, because our hint on the inner loop actually mattered more.

---

Musings:

1134 to 271. I keep staring at those numbers. It was bewildering when I saw it, even though part of me was anticipating something significant.

The thing is... the hardware was always capable of it. The CPU had NEON registers sitting there the entire time, perfectly capable of adding 4 floats at once. It just wasn't being told to. One tiny tag on the loop and suddenly the machine restructures an entire computation.

So much of performance is just permission. The potential is always there. Al it took was someone to say, "go ahead."

Anyhow, wow. I wrote my custom LLVM pass. Feels pretty great, honestly! I hope to begin another compiler adventure soon because I've absolutely been loving learning about all that goes on behind-the-scenes when we hit run.

Until then, off I go to watch more of Agent Kim Reactivated! 😍