Building a Library-First AI Agent SDK in Rust

bigfish — Wed, 07 Oct 2026 14:24:33 +0000

I've been learning about the Pi agent and decided to reimplement it in Rust. What started as a performance experiment turned into something more interesting: a library-first agent SDK that you can embed directly into Rust applications.

This post covers the architecture, the plugin ABI, and a complete minimal example you can run without an API key.

Why a Rust implementation?

Pi is a minimal terminal coding harness. It's designed to be extended and reshaped by the user. But its runtime is TypeScript/Node.

I was running an agent on a low-spec server and kept hitting memory limits. So I ported the core runtime to Rust. But the more useful goal emerged along the way: making the agent runtime embeddable, not just a faster CLI.

If you're building a Rust service that needs to receive tasks, call models, execute custom tools, and stream progress to your own UI, you shouldn't have to spawn a subprocess or treat the terminal as the only entry point.

Library-first architecture

rpi is split into independent crates. Each layer can be used on its own:

Crate	Responsibility
rpi-ai	Unified multi-provider LLM client (Anthropic Messages, OpenAI-compatible, custom gateways)
rpi-agent	Async streaming agent loop with events, hooks, queues, cancellation
rpi-tools	Built-in tools (read, write, edit, bash, grep, find, ls) with a replaceable execution environment
rpi-harness	Session tree, JSONL persistence, context compaction, recovery
rpi-cli	Terminal UI built on the same runtime

If you only need the model and tool loop, use rpi-agent. If you need sessions and recovery, add rpi-harness. The CLI is just one consumer of the same libraries.

A minimal agent in ~100 lines

Here's the complete example. It uses a faux provider — a scripted model that needs no API key and no network. You can run it end-to-end.

Cargo.toml:

[dependencies]
rpi-agent = "0.1"
rpi-ai = "0.1"
tokio = { version = "1", features = ["full"] }
tokio-util = "0.7"
serde_json = "1"
async-trait = "0.1"

main.rs:

use std::io::Write;
use std::sync::Arc;
use rpi_agent::{AgentBuilder, AgentEvent, AgentTool, AgentToolResult, TextContentOrImage};
use rpi_ai::providers::faux::{FauxProvider, FauxScript};
use rpi_ai::types::{Context, Schema, Tool};
use rpi_ai::{Model, Provider, SimpleStreamOptions};
use tokio::runtime::Handle;
use tokio_util::sync::CancellationToken;

// A custom tool: add two integers
struct AddTool { schema: Tool }

impl AddTool {
    fn new() -> Self {
        Self {
            schema: Tool {
                name: "add".to_string(),
                description: "Add two integers a and b.".to_string(),
                parameters: Schema(serde_json::json!({
                    "type": "object",
                    "properties": {
                        "a": { "type": "integer" },
                        "b": { "type": "integer" }
                    },
                    "required": ["a", "b"]
                })),
                constrained_sampling: None,
            },
        }
    }
}

#[async_trait::async_trait]
impl AgentTool for AddTool {
    fn schema(&self) -> &Tool { &self.schema }
    fn label(&self) -> &str { "Add" }

    async fn execute(
        &self,
        _tool_call_id: &str,
        params: serde_json::Value,
        _signal: CancellationToken,
        _on_update: Arc<dyn Fn(rpi_agent::ToolResultPartial) + Send + Sync>,
    ) -> Result<AgentToolResult, rpi_agent::AgentError> {
        let a = params.get("a").and_then(|v| v.as_i64()).unwrap_or(0);
        let b = params.get("b").and_then(|v| v.as_i64()).unwrap_or(0);
        Ok(AgentToolResult::text((a + b).to_string()))
    }
}

// Bridge async Provider to sync StreamFn
fn provider_stream_fn(provider: Arc<dyn Provider>) -> rpi_agent::StreamFn {
    rpi_agent::stream_fn(
        move |model: &Model, ctx: &Context, opts: &SimpleStreamOptions| {
            let provider = Arc::clone(&provider);
            let model = model.clone();
            let ctx = ctx.clone();
            let opts = opts.clone();
            tokio::task::block_in_place(|| {
                Handle::current().block_on(async move {
                    provider.stream_simple(&model, &ctx, &opts).await
                })
            })
        },
    )
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Script the model: first call add, then give the answer
    let script = FauxScript::new()
        .with_tool_call("add", serde_json::json!({ "a": 2, "b": 40 }))
        .with_text("2 + 40 = 42");
    let provider = FauxProvider::new(script);
    let model = provider.default_model().clone();

    let agent = AgentBuilder::new()
        .model(model)
        .system_prompt("You are a minimal agent. Use tools when needed.")
        .tools(vec![Arc::new(AddTool::new())])
        .stream_fn(provider_stream_fn(provider))
        .build()?;

    let mut events = agent.subscribe();

    println!("> user: What is 2 + 40?\n");
    agent.prompt("What is 2 + 40?").await?;

    loop {
        let event = events.recv().await?;
        match event {
            AgentEvent::MessageUpdate {
                assistant_message_event:
                    rpi_ai::types::AssistantMessageEvent::TextDelta { delta, .. },
                ..
            } => {
                print!("{delta}");
                std::io::stdout().flush().ok();
            }
            AgentEvent::ToolExecutionStart { tool_name, args, .. } => {
                println!("→ tool {tool_name}({args})");
            }
            AgentEvent::ToolExecutionEnd { result, is_error, .. } => {
                let tag = if is_error { "tool error" } else { "tool result" };
                println!("← {tag}: {}", result_text(&result));
            }
            AgentEvent::AgentEnd { messages } => {
                println!("\n--- done: {} new message(s) ---", messages.len());
                break;
            }
            _ => {}
        }
    }
    Ok(())
}

fn result_text(result: &AgentToolResult) -> String {
    result.content.iter().map(|c| match c {
        TextContentOrImage::Text(t) => t.text.clone(),
        TextContentOrImage::Image(_) => "[image]".to_string(),
    }).collect::<Vec<_>>().join("\n")
}

Run it with:

cargo run

To use a real model, swap FauxProvider for rpi_ai::providers::anthropic or an OpenAI-compatible provider. The rest of the code stays the same.

Extensions: native Rust cdylibs

rpi extensions are compiled Rust dynamic libraries (.so/.dll/.dylib), loaded at runtime via libloading through a stable C ABI. The contract lives in rpi-plugin-sdk.

The ABI is a hand-defined C ABI because Rust doesn't have a stable ABI. The two sides may be compiled with different Rust versions or crate versions, so no Rust type with a non-C repr or a Drop impl can cross. Owned data crosses as ptr+len with an explicit free function.

There are already 15 extension packages in rpi-package, organized into five categories:

Category	Extensions
Task planning	rpi-todo, rpi-goal, rpi-plan-mode
Human interaction	rpi-ask-user, rpi-permissions
Code analysis	rpi-codegraph, rpi-lens, rpi-subagents
Web tools	rpi-websearch, rpi-webfetch, rpi-firecrawl
Other	rpi-mcp-adapter, rpi-background-tasks, rpi-memory, rpi-voice

Install an extension with:

rpi install rpi-todo

This downloads from crates.io, compiles the cdylib for your platform, and places it in ~/.rpi/agent/extensions. The CLI discovers it automatically.

What's not there yet

Pi ecosystem compatibility. Existing Pi extensions won't run directly. There's a Pi npm/Git bridge in beta, but it's not stable.

It's 0.1.x. The core loop, providers, tools, harness, and CLI all work. The plugin ABI is stable, but if I break it, extensions break.

I reimplemented the Pi agent in Rust: 10 faster startup, 7 less memory

bigfish — Thu, 01 Oct 2026 11:36:58 +0000

I've been learning about the Pi agent and decided to rewrite it in Rust. Now that I've finished a lot of the work, I'd like to show how much faster Rust (rpi) is than TypeScript (Pi). Here are the results:

Metric	rpi (Rust)	Pi (TypeScript)	Difference
`--version`	17.5 ms	169.5 ms	9.7× faster
Cold start	17.7 ms	189.5 ms	10.7× faster
RSS when ready	12.0 MiB	91.5 MiB	7.6× smaller
Install size	21.8 MiB	~385 MiB	~18× smaller

Rust is really cool!

Github:https://github.com/bigfish1913/pi-rust

Rust-native coding-agent runtime

bigfish — Mon, 28 Sep 2026 15:11:45 +0000

Today I finished the implementation of pi-rust. Now I want to share how to build an AI agent.

From my perspective, the core of an agent is the agent loop. The flow looks like this:

Read steering messages
        ↓
Call the model
        ↓
Get assistant message
        ↓
Check for tool calls
        ├─ No: check whether to stop
        └─ Yes: execute tools
                ↓
           Write tool result
                ↓
           Call the model again

A simplified version of the code looks like this:

while has_more_tool_calls || !pending_messages.is_empty() {
    // 1. Inject steering/follow-up messages

    // 2. Call the model and get the assistant response
    let message = stream_assistant_response(...).await?;

    // 3. Check stop_reason
    if message.stop_reason == StopReason::Error {
        ...
    }

    // 4. Extract tool calls from the assistant message
    let tool_calls = message.content.iter()
        .filter_map(|content| match content {
            Content::ToolCall(tool_call) => Some(tool_call.clone()),
            _ => None,
        })
        .collect();

    // 5. Execute tools
    let batch = execute_tool_calls(...).await?;

    // 6. Add ToolResultMessage back to the context
    current_context.messages.push(...);

    // 7. Decide whether to continue to the next round
}

Another question arises: how do we parse messages from different providers, such as Anthropic, OpenAI, and other completion protocols? The response is streamed, so we need to process it in a special way.

while let Some(event) = response.next().await {
    match &event {
        AssistantMessageEvent::Start { partial } => { ... }

        AssistantMessageEvent::TextDelta { partial, .. } => { ... }

        AssistantMessageEvent::ThinkingDelta { partial, .. } => { ... }

        AssistantMessageEvent::ToolCallDelta { partial, .. } => { ... }

        AssistantMessageEvent::Done { .. }
        | AssistantMessageEvent::Error { .. } => { ... }
    }
}

The streamed message sequence can look like this:

Start
  ↓
TextStart
  ↓
TextDelta
  ↓
TextDelta
  ↓
ToolCallStart
  ↓
ToolCallDelta
  ↓
Done

That's the core idea behind pi-rust: keep the agent loop simple, and normalize provider-specific streaming events into one consistent event model.

If you're building something similar, I'd love to hear how you handle provider differences and streaming tool calls.

DEV Community: bigfish