I built a Rust LLM inference engine with custom WGSL GPU kernels, here's what I learned!

saripalli shanmukha kiran sagar — Sat, 30 May 2026 07:33:21 +0000

I've been working on a side project called aether , a Rust LLM inference engine that can load GGUF models and run them with WGPU GPU acceleration.

It started as a way to understand how LLMs actually work under the hood. One thing led to another, and now it has:

Loads GGUF models (Llama/Mistral/Phi/Qwen)
WGPU GPU backend (Metal/Vulkan/DX12)
Custom fused WGSL compute shaders for Q8_0 and Q4_K quantized matmul (dequantize inline instead of a separate pass)
Concurrent request pool for serving multiple users
OpenAI-compatible API server (axum)
Pure Rust, no Python dependencies in the hot path

The GPU path is still experimental (CPU mode is the safe default), but the dequant shaders and the fused matmul kernels were honestly the most fun part to write.

I'm not trying to compete with llama.cpp or MLX, this was primarily a learning project that grew into something actually useful. Happy to answer questions or take feedback.

Stack: Rust, WGPU, WGSL, GGUF, axum, Tokio

https://github.com/theoxfaber/aether

(Full transparency, the majority of this code and post were written with AI assistance. I drove the design decisions, architecture, and testing; AI handled a lot of the implementation. Treat it accordingly.)

I built a pure-Rust browser automation library, no Node.js, no wrappers, just CDP over Tokio

saripalli shanmukha kiran sagar — Sun, 10 May 2026 21:08:14 +0000

I got tired of every Rust browser automation library either being a thin wrapper around Node.js (slow, heavy) or completely unmaintained and archived. So I built ferrous-browser a pure-Rust, async-first Chrome DevTools Protocol client that ships as a single binary.
The core ideas behind it:

Zero Node.js. It talks directly to Chrome over CDP using Tokio WebSockets. No npm, no subprocess bridges, nothing.
Correct multi-page isolation. CDP session IDs are tracked per page so concurrent pages don't leak events into each other — something a lot of existing libraries quietly get wrong.

Race-condition-free event handling. Event handlers are registered before the commands that trigger them, not after. Sounds obvious, but most implementations don't do this.

A Playwright-inspired API. locator(), evaluate(), WaitUntil — familiar if you've used Playwright or Puppeteer, but idiomatic Rust.
Here's the basic setup in Cargo.toml:

ferrous-browser = "0.1"
tokio = { version = "1", features = ["full"] }

Requires Chrome or Chromium installed locally. That's it.
A minimal example, navigate, read a heading, take a screenshot:

use ferrous_browser::{Browser, WaitUntil};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let browser = Browser::launch_chrome(None).await?;
let page = browser.new_page().await?;
page.goto("https://example.com", WaitUntil::Load).await?;
let heading = page.locator("h1").inner_text().await?;
println!("Heading: {heading}");

let png = page.screenshot().await?;
std::fs::write("screenshot.png", png)?;

Ok(())
}

The locator API covers click, type_text, wait_for, inner_text, and get_attribute. evaluate() is generic — you pass a JS expression and it deserializes into whatever Rust type you specify.
For navigation, there are three wait modes: DomContentLoaded for speed, Load for full resource loading, and NetworkIdle which waits until no network activity for 500ms — useful for SPAs.
On errors, every failure carries structured context. No more opaque strings:

match page.goto("https://bad-url", WaitUntil::Load).await {
Err(BrowserError::NavigationFailed { url, reason }) =>
eprintln!("Navigation to {url} failed: {reason}"),
Err(BrowserError::Timeout { operation, secs }) =>
eprintln!("{operation} timed out after {secs}s"),
Err(e) => eprintln!("{e}"),
Ok(_) => {}
}

You can also chain context onto any Result with .context("loading homepage")?.
Compared to the existing Rust options, chromiumoxide (stale), headless_chrome (archived), ferrous-browser is the only one with an active locator API, NetworkIdle support, and structured errors out of the box.

The benchmarks are honest: raw page creation is slower than Puppeteer right now (Chrome's session routing is heavily optimized on their end), but the gap closes fast when the workload is real scraping or E2E testing rather than micro-benchmarks.

What's on the roadmap: cookie management, PDF export, evaluate_handle for remote object references, HAR/trace capture, and full Windows support.

Would love feedback, especially from anyone who's hit the multi-page session isolation bugs in other libraries, that was the main itch I was scratching.

GitHub: https://github.com/theoxfaber/ferrous-browser
Crates.io: https://crates.io/crates/ferrous-browser

DEV Community: saripalli shanmukha kiran sagar

I built a Rust LLM inference engine with custom WGSL GPU kernels, here's what I learned!

I built a pure-Rust browser automation library, no Node.js, no wrappers, just CDP over Tokio