MLIR-to-RTL Simulation Flow: From linalg.matmul to a Systolic Array

Introduction

Hi! I'd like to share the MVP of my hw-mlir-lab project, where I use MLIR to lower matrix multiplication (matmul) to a systolic array simulated in Verilator. This project does not generate RTL from MLIR. Instead, it maps computations to an existing RTL accelerator and validates the compiler-to-hardware path through simulation.

1. Why I'm Building This Project

In modern systems-on-chip, more and more computation is moving to specialized hardware blocks for AI, communications, DSP, image processing, scientific computing, and other workloads. In these systems, the CPU core often focuses on control, while dedicated accelerator IP blocks handle the heavy computation.

But designing a fast hardware block is only half the challenge. The software stack also needs to use it efficiently. Overall performance depends not only on clock frequency and RTL microarchitecture, but also on tiling, padding, data layout, command formats, the runtime interface, memory organization, and how the compiler flow decides which operations to offload to the accelerator.

This creates an awkward gap. The hardware team typically focuses on the RTL block, interfaces, timing, and resource usage. The software team focuses on the compiler, runtime, APIs, and application integration. Yet many decisions sit right between these two worlds. For example, it is difficult to choose a good tile size or data format by looking only at the compiler side or only at the hardware side.

I wanted to build a minimal project that would let me explore the entire chain: from a high-level operation in MLIR to its execution on an RTL model of an accelerator.

The current MVP validates one specific end-to-end path: linalg.matmul in MLIR is transformed to fit a fixed-size 8x8 systolic array, replaced with a custom MLIR operation, lowered to a C ABI call, and executed through a cocotb/Verilator simulation of the RTL block.

MLIR provides a convenient foundation for this compiler flow: start with a high-level operation, apply transformation passes, introduce a hardware-specific operation, and then lower it to a call to a specific accelerator.

Going forward, I want to develop the project into an open framework for prototyping on-chip accelerator subsystems, with a library of MLIR passes, IP blocks, runtime interfaces, RTL simulation, tests, integration with synthesis tools, and exploration of CIRCT.

2. What Works Today and How the MVP Is Structured

The project currently implements one specific end-to-end path: from linalg.matmul in MLIR to matrix multiplication on an RTL model of a systolic array in a cocotb/Verilator simulation.

A simplified view of the flow looks like this:

MLIR input
  |
  v
MLIR transform pipeline
  |
  v
custom standalone dialect / passes
  |
  v
LLVM lowering
  |
  v
native executable
  |
  v
C interface bridge
  |
  v
cocotb / Verilator
  |
  v
SystemVerilog systolic array RTL

The input is an MLIR program containing linalg.matmul. This is a standard high-level linear algebra operation that is not yet tied to any particular hardware block.

The MLIR transformation pipeline then brings it into a form suitable for the current demo accelerator: the matmul is split into 8x8 tiles, with padding applied to handle boundary cases. This is necessary because the RTL systolic array in the MVP expects fixed-size inputs.

Next, matching operations are replaced with a custom MLIR operation:

standalone.systolic_matmul

This operation serves as an explicit marker: this part of the computation must run on an external hardware block rather than as a regular CPU matmul.

For project-specific transformations, I use standalone-opt. This is an extended version of the MLIR standalone example: MLIR provides an example out-of-tree dialect along with an opt-like tool that can serve as a starting point for your own dialect and passes. In my case, I extended this template with the operations and transformations needed to turn matching linalg.matmul operations into calls to the systolic array.

Next, standalone.systolic_matmul is lowered to a standard C-callable interface:

systolic_matmul_8x8(...)

This is an important boundary in the project. To the generated program, the accelerator looks like an ordinary C function. At the moment, that function is backed by a simulation bridge, but a different runtime mechanism could take its place in the future: MMIO, DMA, an FPGA driver, or an interface to a physical ASIC/FPGA prototype.

The current C bridge passes data to the cocotb testbench through a Unix socket. It sends two input tiles of type i8[8][8] and an accumulator of type i32[8][8], then receives the updated result. The cocotb testbench accepts the request, feeds the data into the SystemVerilog systolic array module, waits for the computation to finish, and returns the result to the program.

The MVP therefore explicitly separates three levels:

MLIR level     — representing and transforming the computation
C ABI level    — the boundary between generated code and accelerator calls
RTL level      — executing the operation on a hardware block model

3. How linalg.matmul Becomes a Call to the Systolic Array

Now let's look at the central part of the project: how a regular linalg.matmul gradually becomes a call to a specific hardware block.

The MLIR pipeline in the MVP is divided into several explicit stages. I deliberately save intermediate files after each step so that it is possible to inspect how the IR changes:

01_tiled.mlir
02_padded.mlir
03_bufferized.mlir
04_entry_wrapped.mlir
05_systolic_memref.mlir
06_systolic_call.mlir
07_llvm.mlir
08_llvm.ll

The input is an MLIR program containing linalg.matmul. At this level, the operation knows nothing about the systolic array, RTL, cocotb, or the C ABI. It is simply a high-level description of matrix multiplication.

The first stage is tiling. Using the Transform dialect schedule in matmul_tile.mlir, the original linalg.matmul is split into 8x8x8 tiles. This size is intentional: the current demo systolic array operates on a fixed-size 8x8 block, so the compiler pipeline must bring the computation into a form that the hardware can accept.

The next stage is padding. If the original matrix dimensions are not multiples of 8, boundary tiles appear. The transformation schedule in matmul_pad.mlir handles these cases by padding small or boundary tiles to the fixed size. As a result, the runtime can always send data of the same shape to the systolic array.

Bufferization comes next. Up to this point, the IR may use a tensor-level representation. The one-shot-bufferize pass converts the data to the memref level: the operands are now represented as buffers in memory. This is an important transition because the flow ultimately needs to reach the C ABI and a regular native executable.

A C interface entry wrapper is then created. It allows the C driver—the file containing main—to call the generated MLIR function through a stable external entry point. In the current project, a wrapper is created for matmul_entry; it accepts the input memrefs and an output memref, calls the main MLIR function, and returns the result through the supplied output buffer.

Once the memref-level IR is ready, the project-specific stage begins: conversion to a systolic operation.

The custom pass:

--convert-linalg-matmul-to-systolic

finds matching linalg.matmul operations and replaces them with:

standalone.systolic_matmul

The pass currently expects a specific operation shape:

lhs: memref<8x8xi8>
rhs: memref<8x8xi8>
acc: memref<8x8xi32>

In other words, this is no longer a general matmul of arbitrary size and type. It is a hardware-specific operation prepared for the current RTL block.

The next custom pass is:

--lower-systolic-to-func-call

It removes standalone.systolic_matmul and replaces it with a regular function call:

systolic_matmul_8x8(...)

This is the key point in the entire flow. Before this stage, the operation is still represented in MLIR as an accelerator-specific operation. After this stage, it is a regular call through a C-compatible interface that can be linked with the runtime/interface code.

In brief, the central part of the pipeline looks like this:

linalg.matmul
  |
  | tiling: 8x8x8
  v
tiled linalg.matmul
  |
  | padding
  v
fixed-size matmul tiles
  |
  | bufferization
  v
memref-level IR
  |
  | convert-linalg-matmul-to-systolic
  v
standalone.systolic_matmul
  |
  | lower-systolic-to-func-call
  v
func.call @systolic_matmul_8x8

After that, the remaining MLIR dialects are lowered to the LLVM dialect, and mlir-translate converts it to standard LLVM IR. The compilation pipeline then compiles this LLVM IR into an object file, compiles interface.c, and links everything together with the C driver into a native executable.

At runtime, the systolic_matmul_8x8 function sends the data to the cocotb/Verilator testbench and returns the result.

The generated program therefore looks like a regular native application, but some of its matmul operations are actually executed on an RTL model of the accelerator.

4. Running the Demo and Tests

To make the example easy to run and experiment with, the project is set up to run in Docker: LLVM/MLIR, Clang, Verilator, cocotb, and the custom standalone-opt all run inside a preconfigured environment.

The main demo is launched with a single command:

make demo

This builds the MLIR tools, runs the demo program through the compiler pipeline, compiles a native executable, and launches it alongside the cocotb/Verilator simulation of the RTL systolic array.

After the run, you can inspect the intermediate artifacts from the MLIR pipeline:

build/mlir-pipeline/01_tiled.mlir
build/mlir-pipeline/02_padded.mlir
build/mlir-pipeline/03_bufferized.mlir
build/mlir-pipeline/04_entry_wrapped.mlir
build/mlir-pipeline/05_systolic_memref.mlir
build/mlir-pipeline/06_systolic_call.mlir
build/mlir-pipeline/07_llvm.mlir
build/mlir-pipeline/08_llvm.ll

This is useful for debugging: you can see where linalg.matmul was tiled, where standalone.systolic_matmul appeared, and where it became a regular function call.

To check multiple cases, run the tests:

make test

The tests generate several matmul cases with different dimensions and data patterns, run them through the same flow, and compare the results against the expected outputs.

5. Current Limitations and Future Plans

The current MVP is deliberately limited to a single path: linalg.matmul → 8x8 systolic array → RTL simulation. This makes it possible to validate the entire flow end to end more quickly.

The first limitation is the hardware target itself. The project currently supports a fixed-size systolic array operating on 8x8 tiles. The inputs use i8, while the accumulator and result use i32. This is enough to demonstrate the idea, but it does not yet cover other data types, quantization scenarios, or more complex memory layouts.

The second limitation is the runtime layer. The systolic_matmul_8x8 function currently communicates with the cocotb testbench through a Unix socket. This is a convenient simulation transport for the MVP, but another system could use MMIO, DMA, a driver layer, a command queue, or a different accelerator communication mechanism in its place.

The project also does not yet include a full memory model, an interconnect, a DMA scheduler, a cost model, or automatic selection of operations that would benefit from offloading. The pipeline currently selects only matching linalg.matmul operations that conform to the expected shape after tiling and padding.

There are several directions in which the project could develop.

The first is expanding the set of supported operations and IP blocks. Right now, the only hardware target is the demo systolic array for matmul. Other accelerators could be integrated in the future: DSP blocks, FFT engines, stencil-compute blocks, hash cores, and other computational IP. Each block could have its own MLIR operation, lowering pass, runtime interface, and cocotb/Verilator tests.

The second is a more advanced compiler flow. This includes hardware constraints, a cost model, selection between CPU and accelerator paths, and support for different data types and memory layouts.

The third is the runtime and integration with physical hardware. The socket bridge can be viewed as the initial simulation transport. The next step is a runtime interface that can be backed by MMIO, DMA, or an FPGA prototype.

The fourth is synthesis and CIRCT. The RTL is currently handwritten and verified in simulation. I would like to add a synthesis sanity check to obtain basic resource and timing reports, and explore how CIRCT could help generate wrappers, interfaces, control logic, or other parts of the hardware infrastructure.

More broadly, I want to develop the project into an open framework for prototyping on-chip accelerator subsystems. Not as a replacement for HLS or a ready-made ASIC compiler, but as a reproducible environment for experimenting with compiler passes, IP blocks, runtime interfaces, and RTL simulation within a single flow.

6. Conclusion

I would welcome feedback from anyone working with MLIR/LLVM, RTL, FPGA/ASIC, cocotb/Verilator, EDA tooling, or hardware accelerators. Contributions, issues, architecture discussions, and ideas for new IP blocks or compiler/runtime use cases are all welcome from anyone interested in getting involved.