Building a Rust HTTPS Proxy for AI Agents

#ai #networking #rust #security

Building a Rust HTTPS Proxy for AI Agents

OneCLI's core is an HTTPS man-in-the-middle proxy written in Rust. It intercepts agent HTTP requests, decrypts credentials from an encrypted vault, injects them into request headers, and forwards the request to the target API. All of this happens transparently - the agent doesn't know it's happening.

Building a reliable, performant HTTPS MITM proxy is a surprisingly deep engineering challenge. This post covers the technical decisions we made and the problems we ran into along the way.

Why Rust

The proxy sits in the critical path of every API call an agent makes. Latency matters. Memory safety matters (we're handling decrypted secrets in memory). And we needed strong async I/O to handle many concurrent agent connections without burning through resources.

We considered Go, which would have been fine for the proxy logic, but Rust gave us three things we cared about:

Predictable latency: No garbage collector pauses. When you're adding a hop to every API call, you want sub-millisecond overhead, not occasional 10ms GC stalls.
Memory safety without a runtime: Secrets exist in memory briefly during injection. Rust's ownership model makes it straightforward to reason about when decrypted keys are allocated and dropped. No dangling references to cleartext secrets.
Ecosystem fit: Tokio, Hyper, and Rustls are battle-tested. We're not building TLS or HTTP from scratch - we're composing well-maintained libraries.

The tradeoff is compile times and a steeper learning curve for contributors. We accepted that.

The async foundation: Tokio

The proxy needs to handle hundreds of concurrent connections. Each connection involves at least two TLS handshakes (one from the agent, one to the upstream), header parsing, credential lookup, and forwarding. This is textbook async I/O territory.

We use Tokio as the async runtime with a multi-threaded scheduler. Each incoming connection spawns a task:

let listener = TcpListener::bind("0.0.0.0:10255").await?;

loop {
    let (stream, addr) = listener.accept().await?;
    let vault = vault.clone();

    tokio::spawn(async move {
        if let Err(e) = handle_connection(stream, addr, vault).await {
            tracing::error!(%addr, error = %e, "connection failed");
        }
    });
}

Nothing unusual here. The interesting parts come when we need to handle the HTTPS CONNECT flow.

The CONNECT tunnel

When an HTTP client uses an HTTPS proxy, it doesn't send the full request in plaintext. Instead, it sends a CONNECT request to establish a tunnel:

CONNECT api.openai.com:443 HTTP/1.1
Host: api.openai.com:443
Proxy-Authorization: Basic

In a normal forward proxy, you'd just open a TCP connection to the target and blindly relay bytes. But we need to read and modify the request headers, which means we need to terminate the TLS connection from the agent, inspect the plaintext HTTP request, modify it, then open a new TLS connection to the upstream.

The flow looks like this:

Agent --[TLS]--> OneCLI Proxy --[TLS]--> api.openai.com
         ^                          ^
         |                          |
    Agent trusts               Rustls client
    OneCLI's CA cert           verifies upstream cert

After receiving the CONNECT request, we:

Extract the target hostname from the CONNECT request
Authenticate the agent via the Proxy-Authorization header
Send 200 Connection Established back to the agent
Perform a TLS handshake with the agent using a dynamically generated certificate for the target hostname
Read the now-plaintext HTTP request
Look up and inject credentials
Open a TLS connection to the real target
Forward the modified request
Relay the response back through both TLS layers

Steps 4 through 9 are where the complexity lives.

Dynamic certificate generation

For the agent-side TLS handshake, we need to present a certificate that's valid for the target hostname. We can't use a single wildcard cert - the agent's HTTP client checks that the certificate matches the hostname it's connecting to.

We generate certificates on the fly, signed by a local CA:

fn generate_cert_for_host(
    ca_key: &PrivateKey,
    ca_cert: &Certificate,
    hostname: &str,
) -> Result<(CertifiedKey)> {
    let mut params = CertificateParams::new(vec![hostname.to_string()]);
    params.is_ca = IsCa::NoCa;
    params.not_before = now_utc();
    params.not_after = now_utc() + Duration::hours(24);

    let cert = Certificate::from_params(params)?
        .serialize_der_with_signer(ca_cert, ca_key)?;

    // Return as rustls CertifiedKey
    // ...
}