DEV Community: stevelatif

Simple Firewall with Rust and Aya

stevelatif — Wed, 19 Jun 2024 06:03:54 +0000

Simple Firewall with Rust and Aya

Other Parts in this Series

Part 6 Creating a Simple Firewall

Welcome to Part 6. In this chapter we will extend the work we did in part 5 where we looked at a simple PerCpuArray map to count packets.

Using eBPF we can create a simple firewall/router. With a small amount of code we can drop or redirect packets based on the source and destination addresses. We will implement this in several stages using a hashmap to store the configuration. The initial version will load the IP addresses from user space and to the eBPF kernel code, and with each iteration we can add more functionality.

As before, generate the code using `

cargo generate https://github.com/aya-rs/aya-template

I called the project firewall-001

Modify the generated source code

Modify ebpf firewall-001-ebpf/Cargo.toml to include a dependency for the network-types crate:

[dependencies]
aya-ebpf = "0.1.0"
aya-log-ebpf = "0.1.0"
firewall-001-common = { path = "../firewall-001-common" }
network-types = "0.0.5"

Then modify the ebpf code in firewall-001-ebpf/src/main.rs so we can add HashMap map

In the eBPF code firewall-001-ebpf/src/main.rs the header section should look like this:

    use ayaebpf::{bindings::xdpaction,
               macros::{xdp, 
               map }, // <---- added map macro
               programs::XdpContext,
               maps::HashMap // <--- added hashmaps
               };
    use ayalogebpf::info;
    use core::mem;    // <--- added memory crate
    
    use network_types::{ // Added
        eth::{EthHdr, EtherType}, 
        ip::{IpProto, Ipv4Hdr},
        tcp::TcpHdr,
        udp::UdpHdr,
    };

Add the map definition, as in Part 5 we define the map in the ebpf code in firewall-001/firewall-001-ebpf/src/main.rs

    #[map(name = "SRCIPFILTER")]
    static mut SRCIPFILTER: HashMap<u32, u8> =
        HashMap::<u32, u8>::withmaxentries(1024, 0);

As we are working with the eBPF subsystem in the kernel we will need to work directly with raw pointers. This is where will use the core::mem crate. We need to check the size of data or the verifier will complain

    fn ptr_at<T>(ctx: &XdpContext, offset: usize) -> Result<*const T, ()> {
        let start = ctx.data();
        let end = ctx.data_end();
        let len = mem::size_of::<T>();
        if start + offset + len > end {
            return Err(());
        }
        Ok((start + offset) as *const T)
    }

The packet parsing will be done in the tryfirewall001 function. We will peel off the layers of each packet till we match the rules passed in by the map IP

    let ethhdr: *const EthHdr = ptr_at(&ctx, 0)?; // 
    match unsafe { (*ethhdr).ether_type } {
        EtherType::Ipv4 => {
            info!(&ctx, "received IPv4 packet");
        }
        EtherType::Ipv6 => {
            info!(&ctx, "received IPv6 packet");
            return Ok(xdpaction::XDPDROP);
        }
    
         => return Ok(xdp_action::XDPPASS),
    }

We pass all IPv4 packets but drop any IPv6 packets, in the next section we start to unpack the IPv4 header, first we get the port

    let source_port = match unsafe { (*ipv4hdr).proto } {
        IpProto::Tcp => {
            let tcphdr: *const TcpHdr =
                ptr_at(&ctx, EthHdr::LEN + Ipv4Hdr::LEN)?;
            u16::from_be(unsafe { (*tcphdr).source })
        }
        IpProto::Udp => {
            let udphdr: *const UdpHdr =
                ptr_at(&ctx, EthHdr::LEN + Ipv4Hdr::LEN)?;
            u16::from_be(unsafe { (*udphdr).source })
        }
        _ => return Err(()),
    };

Then we check if the ip address is one in our list of blocked ip addresses

    if unsafe { SRCIP_FILTER.get(&source_addr).issome() } {
        info!(&ctx, "dropping packet ...");
        return Ok(xdpaction::XDPDROP);
    }

The user space code reads a YAML config file that contains a list of IP addresses and an instruction as to what to do to the packets coming from that address.

    ---
    "127.0.0.1" : "block"
    "10.0.0.1"  : "block"
    "10.0.0.2"  : "block"

We will use the figment crate to parse the YAML config file into a hashmap that can be loaded into the eBPF map.

Modify the Cargo.toml file in firewall-001/Cargo.toml to include the dependency:

    figment = { version = "0.10.18", features = ["yaml", "env"] }

And then add the following to the user space rust code in firewall-001/src/main.rs

    use std::net::Ipv4Addr;
    use figment::{Figment, providers::{Yaml, Format}};
    ...
    #[tokio::main]
    async fn main() -> Result<(), anyhow::Error> {
        let opt = Opt::parse();
        let config: HashMap<String,String> = Figment::new()
            .merge(Yaml::file("config.yaml"))
            .extract()?;

Here we extract the config file into a HashMap<String,String> Once we have the entries from our config file in the a HashMap we can load them into the hashmap created in the ebpf code.

This is the opposite of what we did in the Part 5 where we data was stored in the map on the eBPF side and passed to the user space program. Here we load the data from user space and pass it to the eBPF using the map.

    let mut srcip_filter : ayaHashMap<,  u32, u8> =
            ayaHashMap::tryfrom( bpf.map_mut("SRC_IPFILTER").unwrap())?;
    ...
        for (k, v)  in config {
            if v == "block" {
                let addr : Ipv4Addr  = k.parse().unwrap();
                println!("addr {:?}" , addr);
                let  = src_ipfilter.insert(u32::from(addr), 1, 0);
            }
        }

The IP addresses get loaded into the map and are then visible in the eBPF code running in the kernel.

We can use the loopback address 127.0.0.1 to test whether the firewall works First load the eBPF program and attach it to the loopback interface

    RUST_LOG=info cargo xtask run -- -i lo

We can check that it is loaded using bpftool

    $ sudo bpftool prog list | grep -A 5 firewall
    5118: xdp  name firewall_002  tag 64a3874abd9070d2  gpl
            loaded_at 2024-05-01T23:27:54-0700  uid 0
            xlated 7008B  jited 3759B  memlock 8192B  map_ids 1532,1534,1533,1535

We can use the netcat program to test it. In one terminal start a server listening on port 9090

    nc -l 9090

In another terminal send data to the server:

    echo "the quick brown fox jumped over the lazy dog" |  nc 127.0.0.1 9090

In the terminal running the cargo command:

    2024-05-02T06:37:27Z INFO  firewall_002] received IPv4 packet
    [2024-05-02T06:37:27Z INFO  firewall_002] dropping packet ...
    ...

In the netcat server window there will no output showing receipt of a packet

Aya Rust Tutorial part 5: Using Maps

stevelatif — Wed, 19 Jun 2024 00:38:30 +0000

Welcome to part 5. So far we have created a basic hello world program in Part Four. In this chapter we will start looking at how to pass data between the kernel and user space using Maps.

Overview: What is a Map and why do we need them ?

The eBPF verifier enforces a 512 byte limit per stack frame, if you need to handle more data you can store data using maps

Maps are created on the kernel code
Maps are accessible from user space using a system call and a key
Maps allow persistence across program invocations
Maps offer different storage types

Maps in eBPF are a basic building block and we will be using them extensively in the next sections. Our first example will build on the previous example and will be a packet counter using an array.

Let's take a minute to look at the kernel code and see the definitions there. Maps are defined in libbpf.h

struct bpfmapdef {
    unsigned int type;
    unsigned int key_size;
    unsigned int value_size;
    unsigned int max_entries;
    unsigned int map_flags;
};

The different map types:

BPFMAP_TYPEARRAY
BPFMAP_TYPE_PERCPUARRAY
BPFMAP_TYPE_PROGARRAY
BPFMAP_TYPE_PERF_EVENTARRAY
BPFMAP_TYPE_CGROUPARRAY
BPFMAP_TYPE_CGROUPSTORAGE
BPFMAP_TYPE_CGROUPSTORAGE
BPFMAP_TYPE_PERCPU_CGROUPSTORAGE
BPFMAP_TYPEHASH
BPFMAP_TYPE_PERCPUHASH
BPFMAP_TYPE_LRUHASH
BPFMAP_TYPE_LRU_PERCPUHASH
BPFMAP_TYPE_LPMTRIE
BPFMAP_TYPE_STACKTRACE
BPFMAP_TYPE_ARRAY_OFMAPS
BPFMAP_TYPE_HASH_OFMAPS
BPFMAP_TYPE_INODESTORAGE
BPFMAP_TYPE_TASKSTORAGE
BPFMAP_TYPEDEVMAP
BPFMAP_TYPE_DEVMAPHASH
BPFMAP_TYPE_SKSTORAGE
BPFMAP_TYPECPUMAP
BPFMAP_TYPEXSKMAP
BPFMAP_TYPESOCKMAP
BPFMAP_TYPESOCKHASH
BPFMAP_TYPE_REUSEPORTSOCKARRAY
BPFMAP_TYPEQUEUE
BPFMAP_TYPESTACK
BPFMAP_TYPE_STRUCTOPS
BPFMAP_TYPERINGBUF
BPFMAP_TYPE_BLOOMFILTER
BPFMAP_TYPE_USERRINGBUF
BPFMAP_TYPEARENA

are defined in map

The corresponding aya definitions are documented here for the kernel side. The corresponding user space entries are here

Our initial example will be a simple per CPU packet counter that will print out from user space the number of packets arriving at an interface

In the C API helper functions are used on both the kernel and user space side. The helpers have the same names on both sides. The kernel side helper functions have access to a pointer to the map and key.

On the user space side the helper functions use a syscall and file descriptor.

In Aya on the kernel side

array::Array
bloom_filter::BloomFilter
hash_map::HashMap
hash_map::LruHashMap
hash_map::LruPerCpuHashMap
hash_map::PerCpuHashMap
lpm_trie::LpmTrie
percpuarray::PerCpuArray
perf::PerfEventArray
perf::PerfEventByteArray
program_array::ProgramArray
queue::Queue
ring_buf::RingBuf
sock_hash::SockHash
sock_map::SockMap
stack::Stack
stack_trace::StackTrace
xdp::CpuMap
xdp::DevMap
xdp::DevMapHash
xdp::XskMap

While on the user space side We have the same function list.

As before generate the code from the template using the command

 cargo generate https://github.com/aya-rs/aya-template

I called the project xdp-map-counter

Lets set up the packet counter, on the eBPF side:

#![no_std]
#![no_main]
 
use ayaebpf::{bindings::xdpaction,
           macros::{xdp, map},
           programs::XdpContext,
           maps::PerCpuArray,
};
 
const CPU_CORES: u32 = 16;
 
#[map(name="PKTCNTARRAY")]
static mut PACKETCOUNTER: PERCPU<u32> = PerCpuArray::with_max_entries(CPUCORES , 0);
 
#[xdp]
pub fn xdpmap_counter(ctx: XdpContext) -> u32 {
    match tryxdp_mapcounter() {
        Ok(ret) => ret,
        Err() => xdp_action::XDPABORTED,
    }
}
 
#[inline(always)] 
fn tryxdp_mapcounter() -> Result<u32, ()> {
    unsafe {
    let counter = PACKET_COUNTER
            .getptrmut(0)
             .ok_or(())? ;
    *counter += 1;
    }
    Ok(xdpaction::XDPPASS)
}
 
#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}

The map will be created on the eBPF side. We will use a PerCpuArray in this first example. Arrays are simple to work with. With the PerCpuArray each CPU sees its own instance of the map, this means that it avoids lock contention and is therefore the most performant way to get readings from eBPF to user land. The downside is that updating the values from user space can't be done safely.

The size of array must be known at build time so we set a constant with an upper bound on the number of cores on the system const CPU_CORES: u32 = 16

Then we can define a PerCpuArray with CPU_CORES entries initialized to 0.

#[map(name="PKTCNTARRAY")]
static mut PACKETCOUNTER: PerCpuArray<u32> = PerCpuArray::with_max_entries(CPUCORES, 0);

The main work is in the tryxdpcounter function. We get a pointer to the map and then increment the value

    unsafe {
    let counter = PACKET_COUNTER
            .getptrmut(0)
             .ok_or(())? ;
    *counter += 1;
    }

Note that the call to ok_or() is required, failing to have the check here will fail the eBPF verifier.

The packet is then passed on to the networking stack.

    Ok(xdpaction::XDPPASS)

The code on the user space side:

use anyhow::Context;
use aya::programs::{Xdp, XdpFlags};
use aya::{includebytesaligned, Bpf};
use aya::maps::PerCpuValues;
use aya::maps::PerCpuArray;
use aya_log::BpfLogger;
use clap::Parser;
use log::{warn, debug};
use aya::util::nr_cpus;
//use tokio::signal;
 
#[derive(Debug, Parser)]
struct Opt {
    #[clap(short, long, default_value = "eth0")]
    iface: String,
}
 
#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let opt = Opt::parse();
 
    env_logger::init();
    // Bump the memlock rlimit. This is needed for older kernels that don't use the
    // new memcg based accounting, see https://lwn.net/Articles/837122/
    let rlim = libc::rlimit {
        rlimcur: libc::RLIMINFINITY,
        rlimmax: libc::RLIMINFINITY,
    };
    let ret = unsafe { libc::setrlimit(libc::RLIMIT_MEMLOCK, &rlim) };
    if ret != 0 {
        debug!("remove limit on locked memory failed, ret is: {}", ret);
    }
    // This will include your eBPF object file as raw bytes at compile-time and load it at
    // runtime. This approach is recommended for most real-world use cases. If you would
    // like to specify the eBPF program at runtime rather than at compile-time, you can
    // reach for `Bpf::load_file` instead.
    #[cfg(debug_assertions)]
    let mut bpf = Bpf::load(includebytesaligned!(
        "../../target/bpfel-unknown-none/debug/xdp-map-counter"
    ))?;
    #[cfg(not(debug_assertions))]
    let mut bpf = Bpf::load(includebytesaligned!(
        "../../target/bpfel-unknown-none/release/xdp-map-counter"
    ))?;
    if let Err(e) = BpfLogger::init(&mut bpf) {
        // This can happen if you remove all log statements from your eBPF program.
        warn!("failed to initialize eBPF logger: {}", e);
    }
    let program: &mut Xdp = bpf.programmut("xdp_map_counter").unwrap().tryinto()?;
    program.load()?;
    program.attach(&opt.iface, XdpFlags::default())
        .context("failed to attach the XDP program with default flags - try changing XdpFlags::default() to XdpFlags::SKB_MODE")?;
 
    let array = PerCpuArray::tryfrom(bpf.map_mut("PKT_CNTARRAY").unwrap())?;
 
    loop {
    let cc: PerCpuValues<u32> = array.get(&0, 0)?;
    let mut total : u32 =  0;
    //println!("{:?} packets",  cc);
    for ii in 1..nr_cpus().expect("failed to get number of cpus") {
        print!("{} ", cc[ii]);
        total += cc[ii];
    }
    println!("total: {} ", total);
    std::thread::sleep(std::time::Duration::from_secs(1));
    }
    //signal::ctrl_c().await?;    
}

The array reference is created on the user space side here with name 'PKTCNTARRAY'

    let array = PerCpuArray::tryfrom(bpf.map_mut("PKT_CNTARRAY").unwrap())?;

and must match the name declared in the eBPF code

#[map(name="PKTCNTARRAY")]
static mut COUNTER: PerCpuArray<u32> = PerCpuArray::withmax_entries(CPUCORES , 0);

Most of the rest of the code is boilerplate except for the loop at the end which checks every second for the results from the kernel eBPF code and then prints out the stats.

 loop {
    let cc: PerCpuValues<u32> = array.get(&0, 0)?;
    let mut total : u32 =  0;
    for ii in 1..nr_cpus().expect("failed to get number of cpus") {
        print!("{} ", cc[ii]);
        total += cc[ii];
    }
    println!("total: {} ", total);
    std::thread::sleep(std::time::Duration::from_secs(1));
    }

Testing

As we did before we can run it over the loopback interface. Build as before

cargo xtask build-ebpf
cargo build

Then run

cargo xtask run -- -i lo

In another terminal ping the loopback interface


ping 127.0.0.1

In the terminal where you are running the cargo run command you should see

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 total: 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 total: 0 
0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 total: 2 
0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 total: 4 
0 0 6 0 0 0 0 0 0 0 0 0 0 0 0 total: 6 
0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 total: 8 
0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 total: 10

We can see packets arriving and being processed on one core. To run a more stressful test, replace the ping with the following ssh command:

 ssh 127.0.0.1 cat /dev/zero

You should see something like:

 0 0 0 0 0 0 0 0 0 0 0 0 0 0 total: 0 
0 7 6 2 0 0 0 0 0 0 0 5 0 0 0 total: 20 
0 7 6 2 0 0 0 0 0 0 0 6 0 0 0 total: 21 
0 7 6 2 0 0 0 0 0 0 0 6 0 0 0 total: 21 
0 7 6 2 0 0 0 0 0 0 0 6 0 0 0 total: 21 
0 7 6 2 0 0 0 0 0 0 0 6 0 0 0 total: 21 
0 48 12 2 0 16 0 0 0 0 0 8 0 702 0 total: 788 
0 48 133 12 0 527 0 0 0 5 0 8 94 1978 558 total: 3363 
0 48 133 243 0 527 0 17 0 5 0 8 94 1978 3179 total: 6232 
0 48 133 243 0 645 0 144 64 23 0 8 94 1978 5800 total: 9180 
0 48 133 243 0 733 0 201 64 203 2 8 94 1978 8406 total: 12113 
0 48 133 243 0 903 0 333 64 228 2 8 94 1978 11027 total: 15061 
0 48 133 243 0 1447 0 548 64 237 2 8 94 1978 13136 total: 17938 
0 368 133 243 0 3908 0 548 64 237 2 8 94 1978 13136 total: 20719 
0 595 133 416 0 6529 0 548 64 237 2 8 94 1978 13136 total: 23740 
0 683 133 674 0 9294 0 548 64 237 2 8 94 1978 13136 total: 26851 
0 854 133 927 0 11544 0 548 64 296 2 440 94 1978 13136 total: 30016 
...

Run it for a few iterations before Ctrl-C ing it.

As before lets take a minute to look at the eBPF byte code which corresponds to the code in the XDP section of xdp-map-counter/xdp-map-counter-ebpf/src/main.rs

fn tryxdp_mapcounter() -> Result<u32, ()> {
    unsafe {
    let counter = COUNTER
            .getptrmut(0)
             .ok_or(())? ;
    *counter += 1;
    }
    Ok(xdpaction::XDPPASS)
}

Using llvm-objdump as before

$ llvm-obj dump --section=xdp  -S target/bpfel-unknown-none/debug/xdp-map-counter

target/bpfel-unknown-none/debug/xdp-map-counter:    file format elf64-bpf
 
Disassembly of section xdp:
 
0000000000000000 <xdpmapcounter>:
       0:   b7 06 00 00 00 00 00 00 r6 = 0
       1:   63 6a fc ff 00 00 00 00 *(u32 *)(r10 - 4) = r6
       2:   bf a2 00 00 00 00 00 00 r2 = r10
       3:   07 02 00 00 fc ff ff ff r2 += -4
       4:   18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
       6:   85 00 00 00 01 00 00 00 call 1
       7:   15 00 04 00 00 00 00 00 if r0 == 0 goto +4 <LBB0_2>
       8:   61 01 00 00 00 00 00 00 r1 = *(u32 *)(r0 + 0)
       9:   07 01 00 00 01 00 00 00 r1 += 1
      10:   63 10 00 00 00 00 00 00 *(u32 *)(r0 + 0) = r1
      11:   b7 06 00 00 02 00 00 00 r6 = 2
 
0000000000000060 <LBB0_2>:
      12:   bf 60 00 00 00 00 00 00 r0 = r6
      13:   95 00 00 00 00 00 00 00 exit

We see that byte code maps closely to the rust code. After setting up parameters in the first 4 lines, in line 6 we have a call 1 that is a system call to the bpf helper maplookupelem That value gets assigned to r1 where it is incremented on line 9 and is then assigned to a location in memory.

Aya Rust tutorial Part Four XDP Hello World

stevelatif — Sun, 09 Jun 2024 05:05:19 +0000

Aya Rust Tutorial Part 4: XDP Hello World

Welcome to part 4. So far we have installed the prerequisite in part 2,
built eBPF code that loads into the kernel and passes the
verifier. Let's continue on by building another XDP
program that will print a message every time it receives a packet
on an interface. As in part 3 we will use the loopback interface.
This will show how to print a message from the kernel. This is analogous
to using 'bpf_printk' in the eBPF programs in C, see [here](https://github.com/libbpf/libbpf-bootstrap/blob/master/examples/c/kprobe.bpf.c_ul

This will involve only a few more lines of code and
will follow the same build and deployment process as in the previous chapter.

Generating the code

As we did in part 3
generate the code using cargo generate
At the prompt select hello-world as the project name

Using the template, generate the code in directory hello-world\, select the xdp option.

$ cargo generate https://github.com/aya-rs/aya-template  
⚠️   Favorite `https://github.com/aya-rs/aya-template` not found in config, using it as a git repository: https://github.com/aya-rs/aya-template
🤷   Project Name: hello-world
🔧   Destination: /home/steve/articles/learning_ebpf_with_rust/xdp-tutorial/basic01-hello-world/hello-world ...
🔧   project-name: hello-world ...
🔧   Generating template ...
? 🤷   Which type of eBPF program? ›
  cgroup_skb
  cgroup_sockopt
  cgroup_sysctl
  classifier
  fentry
  fexit
  kprobe
  kretprobe
  lsm
  perf_event
  raw_tracepoint
  sk_msg
  sock_ops
  socket_filter
  tp_btf
  tracepoint
  uprobe
  uretprobe
❯ xdp

The generated code will if unaltered behave as a hello world program. In the
first part of this note we will modify the generated code, but come back
to it later
Modify the generated code in the file hello-world/hello-world-ebpf/src/main.rs
so that it looks like:

#![no_std]
#![no_main]

use aya_ebpf::{bindings::xdp_action, macros::xdp, programs::XdpContext};
use aya_ebpf::bpf_printk;

#[xdp]
pub fn hello_world(_ctx: XdpContext) -> u32 {
    unsafe {
        bpf_printk!(b"packet  received!");
    }
xdp_action::XDP_PASS
}

This code uses the unsafe macro bpf_printk
to print out a message every time a packet is received on the interface.
It returns XDP\_PASS\
bpf_printk is a useful tool for debugging. It is globally shared in the kernel
so other programs using it may disrupt its output.

Compile the code

cargo xtask build-ebpf
cargo build

Looking into the BPF-ELF object

As we did in the previous section, lets look at the generated eBPF byte code

$ llvm-readelf --sections target/bpfel-unknown-none/debug/hello-world
There are 7 section headers, starting at offset 0x2e0:

Section Headers:
    [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
    [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
    [ 1] .strtab           STRTAB          0000000000000000 000238 0000a2 00      0   0  1
    [ 2] .text             PROGBITS        0000000000000000 000040 000098 00  AX  0   0  8
    [ 3] xdp               PROGBITS        0000000000000000 0000d8 000030 00  AX  0   0  8
    [ 4] .relxdp           REL             0000000000000000 000228 000010 10   I  6   3  8
    [ 5] .rodata           PROGBITS        0000000000000000 000108 000013 00   A  0   0  1
    [ 6] .symtab           SYMTAB          0000000000000000 000120 000108 18      1   8  8
Key to Flags:
    W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
    L (link order), O (extra OS processing required), G (group), T (TLS),
    C (compressed), x (unknown), o (OS specific), E (exclude),
    R (retain), p (processor specific)

As before we have an xdp section, lets disassemble that:

$ llvm-objdump --no-show-raw-insn --section=xdp  -S target/bpfel-unknown-none/debug/hello-world

target/bpfel-unknown-none/debug/hello-world:    file format elf64-bpf

Disassembly of section xdp:

0000000000000000 <hello_world>:
   0:       r1 = 0 ll
   2:       r2 = 19
   3:       call 6
   4:       r0 = 2
   5:       exit

Recall that the registers for eBPF:

r0: Stores a return value of a function, and exit value for an eBPF program
r1 - R5: Stores function arguments
r6 - R9: For general purpose usage
r10: Stores an address for stack frame

line 0 zeroes out the r1 register
line 2 sets r2 to 19 - the length of the output string
line 3 makes a system call, the mysterious 6 is the index of
bpf_helpers found in bpf.h
line 4 sets the exit value to 2 which corresponds to XDP_PASS

To run this let's use cargo
$ cargo xtask build-ebpf
$ cargo build
$ cargo xtask run -- i lo

To see output, open another terminal enable tracing:

echo 1 | sudo tee /sys/kernel/debug/tracing/tracing_on

Then to see output

sudo cat /sys/kernel/debug/tracing/trace_pipe

From another terminal, ping the loopback interface

ping 127.0.0.1

You should see output being logged in the terminal where you ran the trace_pipe command

$ sudo cat /sys/kernel/debug/tracing/trace_pipe
 ping-75348   [000] ..s21 47214.233803: bpf_trace_printk: packet  received!
 ping-75348   [000] ..s21 47214.233815: bpf_trace_printk: packet  received!
 ping-75348   [007] ..s21 47215.236704: bpf_trace_printk: packet  received!
 ping-75348   [007] ..s21 47215.236737: bpf_trace_printk: packet  received!

Let's return to the previous step where we generated the code. Run the
cargo generate leave the generated code and don't change it

#![no_std]
#![no_main]

use aya_ebpf::{bindings::xdp_action, macros::xdp, programs::XdpContext};
use aya_log_ebpf::info;

#[xdp]
pub fn hello_world(ctx: XdpContext) -> u32 {
    match try_hello_world(ctx) {
        Ok(ret) => ret,
        Err(_) => xdp_action::XDP_ABORTED,
    }
}

fn try_hello_world(ctx: XdpContext) -> Result<u32, u32> {
    info!(&ctx, "received a packet");
    Ok(xdp_action::XDP_PASS)
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}

Build and running it:

cargo xtask build-ebpf
cargo build
RUST_LOG=info cargo xtask run -- -i lo

Then running ping in another terminal:

ping 127.0.0.1

You should see this output in the window where you ran cargo xtask run

[2024-06-08T04:24:17Z INFO  hello_world] Waiting for Ctrl-C...
[2024-06-08T04:24:21Z INFO  hello_world] received a packet
[2024-06-08T04:24:21Z INFO  hello_world] received a packet
[2024-06-08T04:24:22Z INFO  hello_world] received a packet
...

This programs functions in the same way as the first one, but
there are significant differences in the code.

It looks more like idiomatic rust with only one unsafe block in
the panic handler.

However looking at a dump of the byte code:

$ llvm-objdump --section=xdp  -S target/bpfel-unknown-none/debug/hello-world

target/bpfel-unknown-none/debug/hello-world: file format elf64-bpf

Disassembly of section xdp:

0000000000000000 <hello_world>:
   0:   r6 = r1
   1:   r7 = 0
   2:   *(u32 *)(r10 - 4) = r7
   3:   r2 = r10
   4:   r2 += -4
   5:   r1 = 0 ll
   7:   call 1
   8:   if r0 == 0 goto +166 <LBB0_2>
   9:   *(u8 *)(r0 + 2) = r7
  10:   r2 = 11
  11:   *(u8 *)(r0 + 1) = r2
  12:   r1 = 1
  13:   *(u8 *)(r0 + 0) = r1
  14:   r3 = r0
  15:   r3 += 3
  16:   r4 = 0 ll
  18:   r5 = *(u8 *)(r4 + 0)
  19:   *(u8 *)(r3 + 0) = r5
  20:   r5 = *(u8 *)(r4 + 1)
  21:   *(u8 *)(r3 + 1) = r5
  22:   r5 = *(u8 *)(r4 + 2)
  23:   *(u8 *)(r3 + 2) = r5
  24:   r5 = *(u8 *)(r4 + 3)
  25:   *(u8 *)(r3 + 3) = r5
  26:   r5 = *(u8 *)(r4 + 4)
  27:   *(u8 *)(r3 + 4) = r5
  28:   r5 = *(u8 *)(r4 + 5)
  29:   *(u8 *)(r3 + 5) = r5
  30:   r5 = *(u8 *)(r4 + 6)
  31:   *(u8 *)(r3 + 6) = r5
  32:   r5 = *(u8 *)(r4 + 7)
  33:   *(u8 *)(r3 + 7) = r5
  34:   r5 = *(u8 *)(r4 + 8)
  35:   *(u8 *)(r3 + 8) = r5
  36:   r5 = *(u8 *)(r4 + 9)
  37:   *(u8 *)(r3 + 9) = r5
  38:   r5 = *(u8 *)(r4 + 10)
  39:   *(u8 *)(r3 + 10) = r5
  40:   r3 = 3
  41:   *(u8 *)(r0 + 18) = r3
  42:   *(u8 *)(r0 + 17) = r3
  43:   r3 = 2
  44:   *(u8 *)(r0 + 14) = r3
  45:   *(u8 *)(r0 + 20) = r7
  46:   *(u8 *)(r0 + 19) = r2
  47:   *(u8 *)(r0 + 16) = r7
  48:   *(u8 *)(r0 + 15) = r1
  49:   r3 = r0
  50:   r3 += 21
  51:   r5 = *(u8 *)(r4 + 0)
  52:   *(u8 *)(r3 + 0) = r5
  53:   r5 = *(u8 *)(r4 + 1)
  54:   *(u8 *)(r3 + 1) = r5
  55:   r5 = *(u8 *)(r4 + 2)
  56:   *(u8 *)(r3 + 2) = r5
  57:   r5 = *(u8 *)(r4 + 3)
  58:   *(u8 *)(r3 + 3) = r5
  59:   r5 = *(u8 *)(r4 + 4)
  60:   *(u8 *)(r3 + 4) = r5
  61:   r5 = *(u8 *)(r4 + 5)
  62:   *(u8 *)(r3 + 5) = r5
  63:   r5 = *(u8 *)(r4 + 6)
  64:   *(u8 *)(r3 + 6) = r5
  65:   r5 = *(u8 *)(r4 + 7)
  66:   *(u8 *)(r3 + 7) = r5
  67:   r5 = *(u8 *)(r4 + 8)
  68:   *(u8 *)(r3 + 8) = r5
  69:   r5 = *(u8 *)(r4 + 9)
  70:   *(u8 *)(r3 + 9) = r5
  71:   r5 = *(u8 *)(r4 + 10)
  72:   *(u8 *)(r3 + 10) = r5
  73:   *(u8 *)(r0 + 33) = r2
  74:   *(u8 *)(r0 + 34) = r7
  75:   r2 = 4
  76:   *(u8 *)(r0 + 32) = r2
  77:   r3 = r0
  78:   r3 += 35
  79:   r4 = 11 ll
  81:   r5 = *(u8 *)(r4 + 0)
  82:   *(u8 *)(r3 + 0) = r5
  83:   r5 = *(u8 *)(r4 + 1)
  84:   *(u8 *)(r3 + 1) = r5
  85:   r5 = *(u8 *)(r4 + 2)
  86:   *(u8 *)(r3 + 2) = r5
  87:   r5 = *(u8 *)(r4 + 3)
  88:   *(u8 *)(r3 + 3) = r5
  89:   r5 = *(u8 *)(r4 + 4)
  90:   *(u8 *)(r3 + 4) = r5
  91:   r5 = *(u8 *)(r4 + 5)
  92:   *(u8 *)(r3 + 5) = r5
  93:   r5 = *(u8 *)(r4 + 6)
  94:   *(u8 *)(r3 + 6) = r5
  95:   r5 = *(u8 *)(r4 + 7)
  96:   *(u8 *)(r3 + 7) = r5
  97:   r5 = *(u8 *)(r4 + 8)
  98:   *(u8 *)(r3 + 8) = r5
  99:   r5 = *(u8 *)(r4 + 9)
 100:   *(u8 *)(r3 + 9) = r5
 101:   r5 = *(u8 *)(r4 + 10)
 102:   *(u8 *)(r3 + 10) = r5
 103:   *(u8 *)(r0 + 56) = r1
 104:   r1 = 8
 105:   *(u8 *)(r0 + 54) = r1
 106:   r1 = 16
 107:   *(u8 *)(r0 + 49) = r1
 108:   *(u8 *)(r0 + 66) = r7
 109:   *(u8 *)(r0 + 63) = r7
 110:   *(u8 *)(r0 + 62) = r7
 111:   *(u8 *)(r0 + 61) = r7
 112:   *(u8 *)(r0 + 60) = r7
 113:   *(u8 *)(r0 + 59) = r7
 114:   *(u8 *)(r0 + 58) = r7
 115:   *(u8 *)(r0 + 57) = r7
 116:   *(u8 *)(r0 + 55) = r7
 117:   *(u8 *)(r0 + 52) = r7
 118:   *(u8 *)(r0 + 51) = r7
 119:   *(u8 *)(r0 + 50) = r7
 120:   *(u8 *)(r0 + 48) = r7
 121:   *(u8 *)(r0 + 47) = r2
 122:   r1 = 17
 123:   *(u8 *)(r0 + 65) = r1
 124:   *(u8 *)(r0 + 64) = r1
 125:   r1 = 6
 126:   *(u8 *)(r0 + 53) = r1
 127:   r1 = 5
 128:   *(u8 *)(r0 + 46) = r1
 129:   r1 = r0
 130:   r1 += 67
 131:   r2 = 22 ll
 133:   r3 = *(u8 *)(r2 + 0)
 134:   *(u8 *)(r1 + 0) = r3
 135:   r3 = *(u8 *)(r2 + 1)
 136:   *(u8 *)(r1 + 1) = r3
 137:   r3 = *(u8 *)(r2 + 2)
 138:   *(u8 *)(r1 + 2) = r3
 139:   r3 = *(u8 *)(r2 + 3)
 140:   *(u8 *)(r1 + 3) = r3
 141:   r3 = *(u8 *)(r2 + 4)
 142:   *(u8 *)(r1 + 4) = r3
 143:   r3 = *(u8 *)(r2 + 5)
 144:   *(u8 *)(r1 + 5) = r3
 145:   r3 = *(u8 *)(r2 + 6)
 146:   *(u8 *)(r1 + 6) = r3
 147:   r3 = *(u8 *)(r2 + 7)
 148:   *(u8 *)(r1 + 7) = r3
 149:   r3 = *(u8 *)(r2 + 8)
 150:   *(u8 *)(r1 + 8) = r3
 151:   r3 = *(u8 *)(r2 + 9)
 152:   *(u8 *)(r1 + 9) = r3
 153:   r3 = *(u8 *)(r2 + 10)
 154:   *(u8 *)(r1 + 10) = r3
 155:   r3 = *(u8 *)(r2 + 11)
 156:   *(u8 *)(r1 + 11) = r3
 157:   r3 = *(u8 *)(r2 + 12)
 158:   *(u8 *)(r1 + 12) = r3
 159:   r3 = *(u8 *)(r2 + 13)
 160:   *(u8 *)(r1 + 13) = r3
 161:   r3 = *(u8 *)(r2 + 14)
 162:   *(u8 *)(r1 + 14) = r3
 163:   r3 = *(u8 *)(r2 + 15)
 164:   *(u8 *)(r1 + 15) = r3
 165:   r3 = *(u8 *)(r2 + 16)
 166:   *(u8 *)(r1 + 16) = r3
 167:   r1 = r6
 168:   r2 = 0 ll
 170:   r3 = 4294967295 ll
 172:   r4 = r0
 173:   r5 = 84
 174:   call 25

0000000000000578 <LBB0_2>:
 175:   r0 = 2
 176:   exit

So there's a lot here, we will defer a full explanation till later, note that there are now
two system calls on line 7 and line 174:

   7:   call 1  
   ...
 174:   call 25

call 1 corresponds to map_lookup_elem in bpf.h
call 25 corresponding to `perf_event_output in bpf.h

Much of the rest of the byte code is setting up the stack to pass arguments.

Summary

Seen how to set up and deploy a basic hello world program
print out a message when a packet is received
compared two different hello world programs

Aya Rust tutorial Part Three XDP Pass

stevelatif — Thu, 09 May 2024 21:27:37 +0000

Aya Rust Tutorial Part 3: XDP_pass

Assumptions

Assuming you already set up the prerequisites and installed the required tools
and libraries.
The eBPF program will load into the running kernel where it will be
verified.

The eBPF program will run in a virtual machine that is built into the Linux
kernel. This virtual machine has nine general purpose registers R1-R9 and one
read only register R10 that functions as a frame pointer. Running anything
that can be loaded dynamically in the kernel with elevated privileges can
be potential security issue. The eBPF virtual machine contains
a verifier see https://docs.kernel.org/bpf/verifier.html
The verifier will check and reject if the program that contains:

loops
any type of pointer arithmetic
bounds or alignment violations
unreachable instructions

There are other restrictions, consult the documentation for more details

If you have worked with rust code with cargo before, you will have cycled
through iterations of

cargo build
cargo run

Where the source tree of a simple application would like:

$ tree
.
├── Cargo.toml
└── src
    └── main.rs

1 directory, 2 files

and after the application had been compiled:

$ cargo build
   Compiling hello v0.1.0 (/tmp/hello)
    Finished dev [unoptimized + debuginfo] target(s) in 1.07s
$ tree
.
├── Cargo.lock
├── Cargo.toml
├── src
│   └── main.rs
└── target
    ├── CACHEDIR.TAG
    └── debug
        ├── build
        ├── deps
        │   ├── hello-20e1cfc616fb61ac
        │   └── hello-20e1cfc616fb61ac.d
        ├── examples
        ├── hello
        ├── hello.d
        └── incremental
            └── hello-3iozzekpyrysr
                ├── s-gvk87j1cfi-6yflog-9nq5zq3lt8rcg1slvtqhu2l92
                │   ├── 1cwggjlit3xor5e4.o
                │   ├── 3nmgwmthvx9ijli.o
                │   ├── 3qweoew2z2q7s3nh.o
                │   ├── 4j2x83e2uqqcjw4i.o
                │   ├── 4pluyvgu1vsjgq2o.o
                │   ├── 54dgri4zf1sqnocs.o
                │   ├── dep-graph.bin
                │   ├── query-cache.bin
                │   └── work-products.bin
                └── s-gvk87j1cfi-6yflog.lock

9 directories, 18 files

The compiled binary can be found in target/debug/hello and can be run
directly from that location or by using cargo

$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/hello`
Hello, world!

The aya framework creates two programs, an eBPF program that will
be loaded into the kernel and user space program that will be load the
eBPF program, and can also pass and receive data with the eBPF program
using maps - more about these in later parts.

The code framework for the eBPF and user space will be set up using a template.
The eBPF program will be compiled and run using a cargo
workflow extension:

cargo xtask build-ebpf

Compilation is a two stage process:

cargo xtask build-ebpf
cargo build

You can examine the xtask part by looking at the files in xdp_pass/xtask after
you generate the code in the next step:
$ ls xdp-pass/xtask/src/
build_ebpf.rs main.rs run.rs

Generating the code

Why use a template to generate the code? Aya-rs has been rapidly evolving,
using older examples of code with the latest versions can lead to some
errors that are hard to debug. Use the sample code here as a rough guide.
Assuming that we have cargo and the generate extension have been installed, we
can generate the code, at the prompt select xdp-pass as the project name

Using the template, generate the code in directory xdp-pass\, select the xdp option.

$ cargo generate https://github.com/aya-rs/aya-template  
⚠️   Favorite `https://github.com/aya-rs/aya-template` not found in config, using it as a git repository: https://github.com/aya-rs/aya-template
🤷   Project Name: xdp-pass
🔧   Destination: /home/steve/articles/learning_ebpf_with_rust/xdp-tutorial/basic01-xdp-pass/xdp-pass ...
🔧   project-name: xdp-pass ...
🔧   Generating template ...
? 🤷   Which type of eBPF program? ›
  cgroup_skb
  cgroup_sockopt
  cgroup_sysctl
  classifier
  fentry
  fexit
  kprobe
  kretprobe
  lsm
  perf_event
  raw_tracepoint
  sk_msg
  sock_ops
  socket_filter
  tp_btf
  tracepoint
  uprobe
  uretprobe
❯ xdp

The generated files:

$ tree xdp-pass/
xdp-pass/
├── Cargo.toml
├── README.md
├── xdp-pass
│   ├── Cargo.toml
│   └── src
│       └── main.rs
├── xdp-pass-common
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
├── xdp-pass-ebpf
│   ├── Cargo.toml
│   ├── rust-toolchain.toml
│   └── src
│       └── main.rs
└── xtask
    ├── Cargo.toml
    └── src
        ├── build_ebpf.rs
        ├── main.rs
        └── run.rs

8 directories, 13 files

Look at the file: and modify so it looks like:

#![no_std]
#![no_main]

use aya_ebpf::{bindings::xdp_action, macros::xdp, programs::XdpContext};

#[xdp]
pub fn xdp_pass(_ctx: XdpContext) -> u32 {
    xdp_action::XDP_PASS
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    unsafe { core::hint::unreachable_unchecked() }
}

The templated code will run and return XDP\_PASS\

Compile the code

cargo xtask build-ebpf
cargo build

Compile in this order else the cargo build\ will fail.

The xtask step will generate the eBPF object file:
https://raw.githubusercontent.com/stevelatif/articles/main/blogs/target/bpfel-unknown-none/debug/xdp-pass

Looking into the BPF-ELF object

eBPF bytecode is run in a virtual machine in the Linux kernel. There are 10 registers:

R0 stores function return values, and the exit value for an eBPF program
R1-R5 stores function arguments
R6-R9 are for general purpose usage
R10 stores adresses for the stack frame

Inspecting the sections of the eBPF file:

$ llvm-readelf --sections target/bpfel-unknown-none/debug/xdp-pass
There are 5 section headers, starting at offset 0x228:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .strtab           STRTAB          0000000000000000 0001c0 000068 00      0   0  1
  [ 2] .text             PROGBITS        0000000000000000 000040 000098 00  AX  0   0  8
  [ 3] xdp               PROGBITS        0000000000000000 0000d8 000010 00  AX  0   0  8
  [ 4] .symtab           SYMTAB          0000000000000000 0000e8 0000d8 18      1   6  8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  R (retain), p (processor specific)

We can see the xdp section we defined with the xdp macro in the code, so lets inspect that

$ llvm-objdump --no-show-raw-insn --section=xdp  -S target/bpfel-unknown-none/debug/xdp-pass

target/bpfel-unknown-none/debug/xdp-pass:       file format elf64-bpf

Disassembly of section xdp:

0000000000000000 <xdp_pass>:
       0:       r0 = 2
       1:       exit

Setting the r0 register to 2 corresponds to returning XDP_PASS

Use the IOvisor documentation of the opcodes from here https://github.com/iovisor/bpf-docs/blob/master/eBPF.md

We can run the program using cargo, as its an XDP program we will have to specify a network interface.
For this introductory example we can use the the loopback
interface for convenience:

$ RUST_LOG=info cargo xtask run -- -i lo
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/xtask run -- -i lo`
    Finished `dev` profile [optimized] target(s) in 0.10s
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
[2024-04-28T06:14:44Z WARN  xdp_pass] failed to initialize eBPF logger: log event array AYA_LOGS doesn't exist
[2024-04-28T06:14:44Z INFO  xdp_pass] Waiting for Ctrl-C...

We can also load eBPF programs using iproute2

sudo ip link set dev lo xdpgeneric obj   https://raw.githubusercontent.com/stevelatif/articles/main/blogs/target/bpfel-unknown-none/debug/xdp-pass sec xdp

We can see the loaded program with bpftool:

sudo bpftool prog | grep -A 5 xdp
4509: xdp  name xdp_pass  tag 3b185187f1855c4c
        loaded_at 2024-04-28T09:15:17-0700  uid 0
        xlated 16B  jited 22B  memlock 4096B

We can generate a dump of the byte code of the running eBPF byte code using bpftool. Generate a dot file using bpftool and the id number for 4509
from above.

$ sudo bpftool prog dump xlated id 4509 visual &> 4509.dot

And then use that to generate an image file with graphviz

$ dot -Tpng /tmp/4509.dot -o ~/4509.png

The comparable C version of the eBPF looks like:

/* SPDX-License-Identifier: GPL-2.0 */
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
SEC("xdp")
int  xdp_pass_func(struct xdp_md *ctx)
{
    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

Which would be compiled like:

clang -O2 -target bpf -c ebpf_program.c -o ebpf_program.oh

And loaded:

$ sudo  ip link set dev lo xdpgeneric obj   https://raw.githubusercontent.com/stevelatif/articles/main/blogs/xdp_pass.o sec xdp                                                                                                                                                                        
$ sudo ip link show dev lo                                                                                                                                                                                                              
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 xdpgeneric qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000                                                                                                                                                                
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00                                                                                                                                                                                                                          
    prog/xdp id 4529 tag 3b185187f1855c4c jited

Dumping the byte code shows similar sections,

$ llvm-readelf --sections xdp_pass.o
There are 7 section headers, starting at offset 0x108:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .strtab           STRTAB          0000000000000000 0000ba 00004e 00      0   0  1
  [ 2] .text             PROGBITS        0000000000000000 000040 000000 00  AX  0   0  4
  [ 3] xdp               PROGBITS        0000000000000000 000040 000010 00  AX  0   0  8
  [ 4] license           PROGBITS        0000000000000000 000050 000004 00  WA  0   0  1
  [ 5] .llvm_addrsig     LLVM_ADDRSIG    0000000000000000 0000b8 000002 00   E  6   0  1
  [ 6] .symtab           SYMTAB          0000000000000000 000058 000060 18      1   2  8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  R (retain), p (processor specific)

Disassembly is the same as the Rust version:

steve@Peshawer:~/git/aya_discovery/xdp_pass$ llvm-objdump --no-show-raw-insn --section=xdp  -S https://raw.githubusercontent.com/stevelatif/articles/main/blogs/xdp_pass.o

https://raw.githubusercontent.com/stevelatif/articles/main/blogs/xdp_pass.o:   file format elf64-bpf

Disassembly of section xdp:

0000000000000000 <xdp_pass_func_02>:

       0:       r0 = 2

       1:       exit

Summary

We can use Rust with the aya crate to build eBPF programs
Load and unload the eBPF programs we created using
- Cargo
- ip net2
Disassemble the byte code using
- llvm-objdump
Generate graphics to aid in debugging using graphviz.

Aya Rust tutorial Part Two - Setting up

stevelatif — Thu, 09 May 2024 20:31:23 +0000

Part Two: Setting up the Prerequisites

Assumptions

All the examples will be run on Ubuntu Linux. On other distributions your mileage may vary

First step: setup dependencies

Install packages

$ sudo apt install clang llvm libelf-dev libpcap-dev build-essential libc6-dev-i386  \
graphviz  make gcc libssl-dev bc libelf-dev libcap-dev clang gcc-multilib  \
libncurses5-dev git pkg-config libmnl-dev bison flex linux-tools-$(uname -r)

Verify that you have bpftool installed on your system

$ sudo bpftool prog

If there are problems installing it from a package, you can install it from source:

$ git clone --recurse-submodules https://github.com/libbpf/bpftool.git
$ cd bpftool/src
$ make -j$(nproc)
$ sudo https://raw.githubusercontent.com/stevelatif/articles/main/blogs/bpftool prog

Install rust, following the instructions at https://rustup.rs/

$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Once you have rust and cargo installed and in your path, install the following rust related tools:

$ rustup udpate
$ cargo install cargo-generate
$ cargo install bpf-linker
$ cargo install cargo-generate
$ cargo install rustfmt
$ cargo install bpf-linker

Aya Rust tutorial Part One

stevelatif — Thu, 09 May 2024 18:46:22 +0000

Getting started with Aya and eBPF Part 1

Introduction

This is the first in a series of posts looking at eBPF and the
Rust Aya crate. eBPF can run sandboxed programs inside the kernel.
Aya can create the byte code for both the kernel space
programs, and the corresponding userland programs to load the eBPF byte code.

The Extended Berkeley Packet Filter came out as a development on
the Berkeley Packet Filter
dating back to the early 1990s.
BPF based tools tcpdump, ethereal and
the libpcap libraries were used to capture
network packets on the wire, filter them, even inject faults.
Doing anything other than monitoring packets was a time intensive and tricky
operation. More information on eBPF can be found here: https://ebpf.io/

Rust has been around for several years and works well as a system and
general programming language. There are many fine introductions to the language,
a good place to start is here: https://www.rust-lang.org/

In this series of articles we will be looking at
aya-rs, a rust interface to eBPF https://aya-rs.dev/

Two past projects act as motivation for my work with
eBPF and aya.

Scenario 1 SMB Traffic generation

The network interface on
a storage filer was locking up after having
mounted 255 SMB network shares.
To emulate this we had a lab that consisted of:

a linux client box
smbclient, an FTP like interface to Samba
Maxwell Pro from Interworking Labs (iwl.com) to rewrite ethernet headers.

Quick side note on the Maxwell Pro, it
consists of a linux box with two ethernet cards, packets coming in are passed to
a user space application, where they can be modified or have their headers
rewritten, they are then passed out from the other interface.

This had to be thrown together in a few days using whatever was at hand.

Here's a schematic of the setup, bear in mind that this is Linux circa 2004,
Linux networking was more limited in its capabilities.

This would simulate from several hundred, to several thousand windows computers
mounting a network share. The share requests would come in from distinct IP and MAC
addresses.

There are 3 hosts:

a linux client
- Depending on the test, this will have between 300 - 5000 virtual IP addresses, each with a unique IP
An IWL Maxwell Pro to rewrite the ethernet headers (iwl.com)
The device under test: An SMB server

The procedure:

A C wrapper script would run multiple invocations of smbclient
- each smblcient invocation would have it's socket call intercepted and bound to one of the IP addresses using the dynamic linking loader https://man7.org/linux/man-pages/man3/dlopen.3.html
- The Maxwell Pro would rewrite the outgoing MAC address to one based on the IP address
- SMB server would respond to the smbclient request
- The response packets would be intercepted by the Maxwell Pro which would rewrite the SMB server response so that the MAC address was the actual MAC on the linux client

If we were to try to get this working now, could we do it in a more robust manner and not have:

The Maxwell Pro
Manipulating the dynamic linking loader
All the virtual interfaces

Using eBPFs packet parsing and rewriting abilities we could might be able
to do all or some of these.

One of the major headaches with the original project was that it had several disparate
elements implemented in different languages. This was a result of the ad hoc nature of the project.
Being able to build it in a consistent and robust manner would be a major plus.

Scenario 2 Wan Simulation

In this case the ask was to build a wan simulator between two servers to
test acceleration algorithms. The product was a server that might sit in
a remote office and communicate with a similar server in a corporate
data center. Traffic would between the two boxes tunneled and cached
to deal with low bandwidth lossy links.
From the networking perspective this
was more straight forward as it involved using a traffic shaping and
firewalling tool to simulate different WAN scenarios. Most of the work
ended up being in the reporting side of the project.

The traffic shaper/firewall was a FreeBSD tool called dummynet: http://info.iet.unipi.it/~luigi/dummynet/

There were many combinations to test the algorithms in. Stability and consistency
was always a concern.

Why Use eBPF and Aya?

Both these projects had rapid turnarounds from conception to implementation,
but parts of them were quite fragile. My regret was that I didn't have
full access to the networking stack to manipulate packets as they passed
through the stack. eBPF goes a long way to giving us this ability with some
restrictions.
The SMB trafiic generator had small pieces of code in C for running smbclient,
Perl for setting up the networking environment, C++ for running
in the Maxwell Pro to rewrite the ethernet headers. Building and testing
these small programs could only be done on the hosts where they would run
for the most part. Deployment involved using rsync/sftp to move files around.
Not having a consistent build CI pipeline incurs overhead.
There was a similar situation for the WAN emulator. The traffic shaping tool
was part of the FreeBSD kernel which required a kernel rebuild to deploy it
and could only be tested once in place.

Aya allows you to use eBPF, but also build both your userspace and kernel
code with one command using Cargo. Testing and deployments are now
much easier. There are other language bindings to use eBPF, several
of which are relatively mature. Rust does have more stringent requirements
during the compilation phase for code hygiene. This can only be a good thing
when generating code that will be run with escalated privileges.

Next Steps

The first goal will be to learn enough about Aya and eBPF to see if we can
implement, or at least think of implementing some parts of the traffic
generator and traffic shaper. These are esoteric tools
but the ideas that we will need for implementing them will mean that
we can talk about

firewalls
load balancers
networking protocols