DEV Community

Samar lassoued
Samar lassoued

Posted on

Cross-Compiling Linux Kernel Module: A Hands-On Guide

#embedded #development #linux #crosscompilation

Welcome to my first dev.to post! today, we're diving into the practical world of cross-compiling Linux kernel modules, an essential skill for embedded systems development. Whether you're working with Raspberry Pi, custom ARM boards, or virtualized environments, understanding this process is crucial.

Before we get our hands dirty with code, let's clarify some fundamental concepts. If you've worked with IDEs, you've likely compiled code with a button press without seeing the underlying process. Linux users might be more familiar with command-line tools like gcc or make.

Compilation Process

Consists of transforming human-readable source code into machine-executable binaries through four main stages:

  1. Preprocessing: Expands macros and handles includes
  2. Compilation : A tool called compiler takes the output of the preprocessor and converts it to assembly language.
  3. Assembly : Translates to machine code (object files)
  4. Linking : Combines all object code into executable binaries

Understanding Compilation Strategies

1. Native Compilation

  • Definition: It consists of compiling software on the same architecture and system where it will run.

2. Cross Compilation

  • Definition: It consists of compiling software on a different architecture than the one it will run on, using a cross-compilation toolchain. So the final binary output will run on different target machine.

Why Cross-Compilation?

We need Cross compilation since we can't compile directly on the target itself due to limited ressources and the compilation process itself needs large ressources.

The Cross-Compilation Toolchain

A toolchain includes compilers, linkers, and libraries needed for building software. For ARM64 development, we use the aarch64-linux-gnu toolchain:

  • Compiler: aarch64-linux-gnu-gcc
  • Binutils: aarch64-linux-gnu-ld, aarch64-linux-gnu-objcopy...
  • Libraries: ARM64 versions of standard C libraries

This toolchain runs on our x86-64 host but produces ARM64 binaries.

Ready to start? let's go!

1. Environment Setup

Let's start by setting up our development environment. First, we have to make sure that we have all the necessary tools installed:

sudo apt install -y \
    qemu-system-arm \
    gcc-aarch64-linux-gnu \
    build-essential \
    libncurses-dev \
    bison \
    flex \
    libssl-dev \
    libelf-dev \
    cpio \
    kmod
Enter fullscreen mode Exit fullscreen mode

Note: I am running a system based on ubuntu 22.04.

After that, we need to Download the github project that contains the code source of the linux kernel module(hello_module.c + Makelfile):

git clone https://github.com/samar12lassoued/kernel-module-arm-qemu
Enter fullscreen mode Exit fullscreen mode

Note: It's a Github repo where i put all the source code of this tutorial.

2. Kernel source downloading and preparation 

# Download Linux kernel 6.6
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.6.tar.xz
tar xf linux-6.6.tar.xz
cd linux-6.6
Enter fullscreen mode Exit fullscreen mode

We need now to specify the target architecture and the cross-compiler prefix:

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-
Enter fullscreen mode Exit fullscreen mode

3. Kernel Configuration and build

Configure with default settings: 

make defconfig
Enter fullscreen mode Exit fullscreen mode

it Creates a default configuration file (.config) for our target architecture.

Optional:

we can Customize kernel options and this command opens a text-based menu interface to customize kernel options:

make menuconfig
Enter fullscreen mode Exit fullscreen mode

How to navigate:

  • Arrow keys: Move between options
  • Enter: Select/submenu
  • Y: Enable option
  • N: Disable option
  • M: Compile as module
  • /: Search

Kernel Configuration Verification

The great news is that ARM64 defconfig already includes all essential options for our example to boot with QEMU. We can verify that some critical options are enabled:

grep -E "CONFIG_SERIAL_AMBA_PL011|CONFIG_BLK_DEV_INITRD|CONFIG_MODULES|CONFIG_DEVTMPFS|CONFIG_PROC_FS|CONFIG_SYSFS" .config
Enter fullscreen mode Exit fullscreen mode

Output Shows everything we need:

CONFIG_BLK_DEV_INITRD=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_MODULES=y
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_DEVTMPFS_SAFE is not set
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_PROC_FS=y
CONFIG_SYSFS=y
Enter fullscreen mode Exit fullscreen mode

Build kernel image

make -j$(nproc) Image
make -j$(nproc) modules
Enter fullscreen mode Exit fullscreen mode

Verify the build

file arch/arm64/boot/Image
Enter fullscreen mode Exit fullscreen mode

Should show: Linux kernel ARM64 boot executable Image

4. BusyBox Compilation for Minimal RootFS

As we all know a Linux Kernel itself can't function, it requires a root filesystem containing all userspace utilities and initialization scripts. This rootfs provides the environment where user processes run and system services begin.

For our embedded environment, we'll build a minimal rootfs using BusyBox, which packages numerous essential Unix utilities into a single compact binary. This approach dramatically simplifies our build by creating a statically linked system, eliminating the need for dynamic linking or shared library dependencies.

cd ..
wget https://busybox.net/downloads/busybox-1.36.0.tar.bz2
tar xf busybox-1.36.0.tar.bz2
cd busybox-1.36.0

# Configure for static linking
make defconfig
make menuconfig
Enter fullscreen mode Exit fullscreen mode

Enable: Settings → Build static binary (no shared libs)

# Cross-compile BusyBox
make CROSS_COMPILE=aarch64-linux-gnu- install
Enter fullscreen mode Exit fullscreen mode

Verify ARM64 binary:

file _install/bin/busybox
Enter fullscreen mode Exit fullscreen mode

5. Root FileSystem Contruction 

cd ..
# Create directory structure
mkdir -p rootfs/{bin,dev,etc,proc,sys,tmp,usr/bin}

# Copy BusyBox
cp -r busybox-1.36.0/_install/* rootfs/

# Create essential device nodes
sudo mknod rootfs/dev/console c 5 1
sudo mknod rootfs/dev/null c 1 3
Enter fullscreen mode Exit fullscreen mode

There are a number of ways to boot into a linux kernel. Initramfs is commonly used in modern linux distributions. It is a file that is either embedded in the kernel or adjacent to the kernel. It is loaded completely into volatile memory and its purpose is to start up system required hardware devices and drives.
In our case, we're going to simply use initramfs as a method to boot our system into a volatile state. This way, all changes that we make to the environment will be wiped upon reboot

6. Init Script Creation

For the kernel to pass control to the userspace, it attempts to execute an init process as the first process. Therefore we'll create our own shell script to run as the first process. Create rootfs/init script:

cd rootfs
cat > init << 'EOF'
#!/bin/sh

# Mount essential filesystems
mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev

# Set up environment
export PATH=/bin:/sbin:/usr/bin:/usr/sbin
export SHELL=/bin/sh

echo "Minimal Linux environment ready"
exec /bin/sh
EOF

# Setup the script to be executable
chmod +x init
cd ..
Enter fullscreen mode Exit fullscreen mode

7. Module Compilation and Integration 

# Cross-compile the module
make KDIR=linux-6.6 cross-compile

# Verify ARM64 module
file hello_module.ko

# Add to rootfs and rebuild initramfs
cp hello_module.ko rootfs/
cd rootfs
find . | cpio -o -H newc | gzip > ../initramfs.cpio.gz
cd ..
Enter fullscreen mode Exit fullscreen mode

8. QEMU Testing and Validation

Now we can test our fully emulated system to see if it boots:

qemu-system-aarch64 \
    -M virt \
    -cpu cortex-a53 \
    -smp 2 \
    -m 1G \
    -kernel linux-6.6/arch/arm64/boot/Image \
    -initrd initramfs.cpio.gz \
    -append "console=ttyAMA0 earlycon=pl011,0x9000000 init=/init" \
    -nographic \
    -no-reboot
Enter fullscreen mode Exit fullscreen mode

This QEMU command launches a 64-bit ARM virtual machine using the generic virt board, emulating a Cortex-A53 CPU with 2 cores and 1 GB of RAM. It boots a Linux 6.6 kernel with an initramfs as the root filesystem, passes kernel parameters to use the PL011 UART (ttyAMA0) for console output, and starts /init as the first user process. The -nographic option routes all input/output to the terminal instead of a graphical window, and -no-reboot ensures QEMU exits instead of restarting on shutdown or crash.

9.Runtime Module Testing 

# Load and test module
insmod hello_module.ko
dmesg | tail -3

rmmod hello_module
dmesg | tail -3
Enter fullscreen mode Exit fullscreen mode

See Embedded Linux is not that scary after all ! so what we have achieved today :

  1. Cross-compiled a Linux kernel for ARM64
  2. Built a minimal root filesystem with BusyBox
  3. Created a working initramfs
  4. Wrote and cross-compiled a kernel module
  5. Booted a virtual ARM64 machine with QEMU
  6. Loaded and tested our custom kernel module

🌟 Remember All you have to do is start learning and implementing things you thought were hard or complex.🌟

And stay tuned for my next post!

Additional Ressources:

Linux Kernel documentation
BusyBox Official Documentation
QEMU ARM System Emulation
Building External Modules Guide
The Linux Kernel Module Programming Guide
ARM gcc cross compiler

Top comments (0)