DEV Community

Bartosz Osiej
Bartosz Osiej

Posted on

I built an eBPF ransomware detector in Rust

I built an eBPF ransomware detector in Rust

Kernel-level tracing, sliding-window alerts, and a cyberpunk TUI — in 1.7MB.

Why?

Ransomware detection typically requires expensive enterprise solutions. I wanted something that:

  1. Runs at the kernel level — catches file operations before they hit disk
  2. Works in real-time — no post-mortem analysis, immediate alerts
  3. Ships as a single binary — no agents, no daemons, no dependencies
  4. Is open source — auditable, free, and community-driven

That's Halcyon Process Monitor.

The core idea

Ransomware encrypts files rapidly. A normal user opens maybe 10-20 files per second. Ransomware? Hundreds or thousands.

Halcyon traces every openat syscall at the kernel level using eBPF, maintains a 1-second sliding window per process, and alerts when any process exceeds a configurable threshold:

Process opens 50+ files in 1 second → ALERT
Enter fullscreen mode Exit fullscreen mode

Simple. Effective. No machine learning needed.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        KERNEL SPACE (eBPF)                      │
│                                                                 │
│  sys_enter_execve ──┐                                           │
│  sys_enter_openat  ──┤                                           │
│  sys_enter_connect ──┤   ProcessEvent    PerfEventArray         │
│  sys_enter_accept  ──┼──► (map)    ────► (per-CPU buffers)     │
│  sys_enter_sendto  ──┤                                           │
│  sys_enter_recvfrom ─┘                                           │
└──────────────────────────────────┬──────────────────────────────┘
                                   │
┌──────────────────────────────────▼──────────────────────────────┐
│                     USERSPACE (Rust)                            │
│                                                                 │
│  reader thread ──► channel ──► Monitor ──► TUI / JSON / Web    │
│       │                  │         │                            │
│       │                  │    sliding window                    │
│       │                  │    + alerting                        │
│       │                  │    + process tree                    │
│       │                  │    + file ranking                    │
│       │                  │    + network tracking                │
│       │                  │    + heatmap                         │
│       └──► perf buffer   └──► search/filter                    │
└─────────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The kernel side is written in #![no_std] Rust using aya, and compiles to eBPF bytecode. The userspace side uses aya's userspace libraries to load the programs and read events.

The eBPF program

Here's the kernel-side code that traces openat syscalls:

// process-monitor-ebpf/src/main.rs
#[kprobe]
pub fn sys_enter_openat(ctx: ProbeContext) -> u32 {
    let pid = bpf_get_current_pid_tgid() >> 32;
    let uid = bpf_get_current_uid_gid() as u32;

    // Read comm (process name)
    let mut comm = [0u8; 16];
    unsafe {
        bpf_get_current_comm(&mut comm as *mut _ as *mut _, 16);
    }

    // Read filename from registers (x86_64: rsi = filename)
    let mut filename = [0u8; 256];
    let ptr = ctx.arg(1) as usize;
    unsafe {
        bpf_probe_read_user_str(
            &mut filename as *mut _ as *mut _,
            256,
            ptr as *const _,
        );
    }

    // Send event to userspace
    let event = ProcessEvent {
        pid,
        uid,
        kind: EventKind::Open,
        comm,
        filename,
    };

    unsafe {
        EVENTS.output(&ctx, &event, 0);
    }

    0
}
Enter fullscreen mode Exit fullscreen mode

Key points:

  • bpf_probe_read_user — never dereference userspace pointers directly (eBPF verifier rejects that)
  • PerfEventArray — per-CPU buffers avoid lock contention
  • Fixed-size structs — no allocations in kernel code

The sliding window

The real intelligence is in userspace. The monitor maintains a rolling 1-second window per PID:

struct SlidingWindow {
    events: VecDeque<Instant>,
    threshold: u32,
}

impl SlidingWindow {
    fn record(&mut self) -> Option<Alert> {
        let now = Instant::now();
        self.events.push_back(now);

        // Remove events older than 1 second
        while self.events.front().map_or(false, |t| now.duration_since(*t) > Duration::from_secs(1)) {
            self.events.pop_front();
        }

        if self.events.len() >= self.threshold as usize {
            Some(Alert {
                pid: self.pid,
                opens_in_1s: self.events.len() as u32,
                timestamp: now,
            })
        } else {
            None
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

50 files per second triggers an alert. You can configure this with --alert-threshold.

The TUI

The terminal interface has 7 panels:

Panel What it shows
EVENTS Live event log with search/filter
PROCESSES Hierarchical process tree with mini-bars
NETWORK Real-time connections (connect/accept/send/recv)
TOP FILES Most-opened files with Shannon entropy scores
FILE TYPES Extension frequency with colored bars
ALERTS Alert history with timestamps
HEATMAP Syscall frequency visualization

Built with ratatui, styled in cyberpunk colors.

What else it traces

Halcyon isn't just about file opens. It traces 6 syscalls:

Syscall Why
execve Process creation — spot suspicious binaries
openat File access — core ransomware detection
connect Outbound connections — spot C2 callbacks
accept Inbound connections — spot reverse shells
sendto Data exfiltration
recvfrom Data reception

Network tracing

New in v0.4 — Halcyon also tracks network activity:

{"type": "connect", "pid": 1234, "comm": "curl", "file": "93.184.216.34:443"}
{"type": "alert", "pid": 2126, "comm": "Cache2 I/O", "opens_in_1s": 50}
Enter fullscreen mode Exit fullscreen mode

This catches ransomware that exfiltrates data before encrypting.

Deployment options

Single binary

sudo process-monitor
Enter fullscreen mode Exit fullscreen mode

Web dashboard

sudo process-monitor --web 0.0.0.0:8080
Enter fullscreen mode Exit fullscreen mode

REST API + WebSocket + Prometheus metrics.

Kubernetes

kubectl apply -f k8s/  # DaemonSet on every node
Enter fullscreen mode Exit fullscreen mode

Go agent

./halcyon-agent watch  # WebSocket live events
Enter fullscreen mode Exit fullscreen mode

C FFI

#include "halcyon.h"
halcyon_monitor_t* monitor;
halcyon_monitor_create("/path/to/bpf.o", 50, &monitor);
Enter fullscreen mode Exit fullscreen mode

Build sizes

Variant Size
TUI-only 1.7MB
Web-featured 2.5MB

Full LTO, panic = "abort", symbol stripping. A single static binary.

Stats

Metric Count
Syscalls traced 6
TUI panels 7
Build variants 3 (TUI, web, both)
Binary size 1.7MB (TUI)
Deployment targets 5 (binary, web, k8s, Go agent, C FFI)

What I learned

  1. eBPF is surprisingly accessible — with aya, you write Rust, not C. The verifier is strict but fair.
  2. Per-CPU buffers are essential — shared maps cause contention. PerfEventArray with per-CPU buffers scales to millions of events per second.
  3. The sliding window is the hard part — kernel tracing is straightforward; the intelligence is in userspace.
  4. Rust's zero-cost abstractions matter — a 1.7MB binary with a full TUI, eBPF loader, and sliding window. No runtime overhead.

Try it

# Install
git clone https://github.com/BartoszOsiej/halcyon-process-monitor.git
cd halcyon-process-monitor
./build.sh

# Run (requires root + Linux 5.8+)
sudo target/release/process-monitor-tui
Enter fullscreen mode Exit fullscreen mode

Or Docker:

docker run --privileged -v /sys/kernel/btf:/sys/kernel/btf \
    halcyon-process-monitor process-monitor
Enter fullscreen mode Exit fullscreen mode

Built by Bartosz Osiej — 19, Poland, open to first paid role. Everything here is deployed infrastructure, not a tutorial.

Star the repo: github.com/BartoszOsiej/halcyon-process-monitor

Top comments (0)