bigfish

Posted on Oct 7 Originally published at blog.laofu.online

Building a Library-First AI Agent SDK in Rust

#ai #opensource #programming #rust

I've been learning about the Pi agent and decided to reimplement it in Rust. What started as a performance experiment turned into something more interesting: a library-first agent SDK that you can embed directly into Rust applications.

This post covers the architecture, the plugin ABI, and a complete minimal example you can run without an API key.

Why a Rust implementation?

Pi is a minimal terminal coding harness. It's designed to be extended and reshaped by the user. But its runtime is TypeScript/Node.

I was running an agent on a low-spec server and kept hitting memory limits. So I ported the core runtime to Rust. But the more useful goal emerged along the way: making the agent runtime embeddable, not just a faster CLI.

If you're building a Rust service that needs to receive tasks, call models, execute custom tools, and stream progress to your own UI, you shouldn't have to spawn a subprocess or treat the terminal as the only entry point.

Library-first architecture

rpi is split into independent crates. Each layer can be used on its own:

Crate	Responsibility
rpi-ai	Unified multi-provider LLM client (Anthropic Messages, OpenAI-compatible, custom gateways)
rpi-agent	Async streaming agent loop with events, hooks, queues, cancellation
rpi-tools	Built-in tools (read, write, edit, bash, grep, find, ls) with a replaceable execution environment
rpi-harness	Session tree, JSONL persistence, context compaction, recovery
rpi-cli	Terminal UI built on the same runtime

If you only need the model and tool loop, use rpi-agent. If you need sessions and recovery, add rpi-harness. The CLI is just one consumer of the same libraries.

A minimal agent in ~100 lines

Here's the complete example. It uses a faux provider — a scripted model that needs no API key and no network. You can run it end-to-end.

Cargo.toml:

[dependencies]
rpi-agent = "0.1"
rpi-ai = "0.1"
tokio = { version = "1", features = ["full"] }
tokio-util = "0.7"
serde_json = "1"
async-trait = "0.1"

main.rs:

use std::io::Write;
use std::sync::Arc;
use rpi_agent::{AgentBuilder, AgentEvent, AgentTool, AgentToolResult, TextContentOrImage};
use rpi_ai::providers::faux::{FauxProvider, FauxScript};
use rpi_ai::types::{Context, Schema, Tool};
use rpi_ai::{Model, Provider, SimpleStreamOptions};
use tokio::runtime::Handle;
use tokio_util::sync::CancellationToken;

// A custom tool: add two integers
struct AddTool { schema: Tool }

impl AddTool {
    fn new() -> Self {
        Self {
            schema: Tool {
                name: "add".to_string(),
                description: "Add two integers a and b.".to_string(),
                parameters: Schema(serde_json::json!({
                    "type": "object",
                    "properties": {
                        "a": { "type": "integer" },
                        "b": { "type": "integer" }
                    },
                    "required": ["a", "b"]
                })),
                constrained_sampling: None,
            },
        }
    }
}

#[async_trait::async_trait]
impl AgentTool for AddTool {
    fn schema(&self) -> &Tool { &self.schema }
    fn label(&self) -> &str { "Add" }

    async fn execute(
        &self,
        _tool_call_id: &str,
        params: serde_json::Value,
        _signal: CancellationToken,
        _on_update: Arc<dyn Fn(rpi_agent::ToolResultPartial) + Send + Sync>,
    ) -> Result<AgentToolResult, rpi_agent::AgentError> {
        let a = params.get("a").and_then(|v| v.as_i64()).unwrap_or(0);
        let b = params.get("b").and_then(|v| v.as_i64()).unwrap_or(0);
        Ok(AgentToolResult::text((a + b).to_string()))
    }
}

// Bridge async Provider to sync StreamFn
fn provider_stream_fn(provider: Arc<dyn Provider>) -> rpi_agent::StreamFn {
    rpi_agent::stream_fn(
        move |model: &Model, ctx: &Context, opts: &SimpleStreamOptions| {
            let provider = Arc::clone(&provider);
            let model = model.clone();
            let ctx = ctx.clone();
            let opts = opts.clone();
            tokio::task::block_in_place(|| {
                Handle::current().block_on(async move {
                    provider.stream_simple(&model, &ctx, &opts).await
                })
            })
        },
    )
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Script the model: first call add, then give the answer
    let script = FauxScript::new()
        .with_tool_call("add", serde_json::json!({ "a": 2, "b": 40 }))
        .with_text("2 + 40 = 42");
    let provider = FauxProvider::new(script);
    let model = provider.default_model().clone();

    let agent = AgentBuilder::new()
        .model(model)
        .system_prompt("You are a minimal agent. Use tools when needed.")
        .tools(vec![Arc::new(AddTool::new())])
        .stream_fn(provider_stream_fn(provider))
        .build()?;

    let mut events = agent.subscribe();

    println!("> user: What is 2 + 40?\n");
    agent.prompt("What is 2 + 40?").await?;

    loop {
        let event = events.recv().await?;
        match event {
            AgentEvent::MessageUpdate {
                assistant_message_event:
                    rpi_ai::types::AssistantMessageEvent::TextDelta { delta, .. },
                ..
            } => {
                print!("{delta}");
                std::io::stdout().flush().ok();
            }
            AgentEvent::ToolExecutionStart { tool_name, args, .. } => {
                println!("→ tool {tool_name}({args})");
            }
            AgentEvent::ToolExecutionEnd { result, is_error, .. } => {
                let tag = if is_error { "tool error" } else { "tool result" };
                println!("← {tag}: {}", result_text(&result));
            }
            AgentEvent::AgentEnd { messages } => {
                println!("\n--- done: {} new message(s) ---", messages.len());
                break;
            }
            _ => {}
        }
    }
    Ok(())
}

fn result_text(result: &AgentToolResult) -> String {
    result.content.iter().map(|c| match c {
        TextContentOrImage::Text(t) => t.text.clone(),
        TextContentOrImage::Image(_) => "[image]".to_string(),
    }).collect::<Vec<_>>().join("\n")
}

Run it with:

cargo run

To use a real model, swap FauxProvider for rpi_ai::providers::anthropic or an OpenAI-compatible provider. The rest of the code stays the same.

Extensions: native Rust cdylibs

rpi extensions are compiled Rust dynamic libraries (.so/.dll/.dylib), loaded at runtime via libloading through a stable C ABI. The contract lives in rpi-plugin-sdk.

The ABI is a hand-defined C ABI because Rust doesn't have a stable ABI. The two sides may be compiled with different Rust versions or crate versions, so no Rust type with a non-C repr or a Drop impl can cross. Owned data crosses as ptr+len with an explicit free function.

There are already 15 extension packages in rpi-package, organized into five categories:

Category	Extensions
Task planning	rpi-todo, rpi-goal, rpi-plan-mode
Human interaction	rpi-ask-user, rpi-permissions
Code analysis	rpi-codegraph, rpi-lens, rpi-subagents
Web tools	rpi-websearch, rpi-webfetch, rpi-firecrawl
Other	rpi-mcp-adapter, rpi-background-tasks, rpi-memory, rpi-voice

Install an extension with:

rpi install rpi-todo

This downloads from crates.io, compiles the cdylib for your platform, and places it in ~/.rpi/agent/extensions. The CLI discovers it automatically.

What's not there yet

Pi ecosystem compatibility. Existing Pi extensions won't run directly. There's a Pi npm/Git bridge in beta, but it's not stable.

It's 0.1.x. The core loop, providers, tools, harness, and CLI all work. The plugin ABI is stable, but if I break it, extensions break.

DEV Community