Zero-Copy Webhook Parsing: Optimizing Rust and Go Ingress Pipelines for 100k RPS

Imagine a webhook ingestion service, such as a durable intake layer like InstaWebhook that accepts events, stores them and queues delivery work outside the request path. Suppose it has to absorb bursts of 100,000 requests per second (RPS). At that rate, small per-request costs add up quickly. A parser that makes 15 allocations per request produces 1.5 million allocations per second, which the allocator, and in Go the garbage collector, has to absorb.

This article is about the parsing step of the pipeline. It shows how to verify signatures before parsing, borrow from the request buffer in Rust with serde, cut allocations in Go with easyjson, and what has changed in Go’s standard library as of Go 1.27. The 100k RPS figure is a design target for illustration, not a benchmark result. Measure your own payloads before drawing conclusions.

1. Where the Allocations Come From

A conventional parse lifecycle has several hidden costs:

1. Buffering the body. The request body has to be read into a contiguous buffer. In Go, io.ReadAll grows a fresh slice on every call. In Rust frameworks, the body is collected into a buffer as well.
2. Per-field allocations. A naive decoder allocates an owned String (Rust) or string (Go) for every key and value it keeps.
3. Reflection and interface boxing (Go). Go’s encoding/json historically inspects struct fields and types at runtime, which costs CPU and allocations.
4. Parsing before authenticating. If you build objects from an unauthenticated body, you spend CPU and memory on traffic you were going to reject anyway.

You can see the allocation effect in the benchmarks published by the easyjson project, which compare it against encoding/json. These numbers come from the project’s README, were measured with Go 1.6 (2016), and use a Twitter API sample. The “small” case is an 80-byte slice of that sample and the “regular” case is the full 13 kB document.

Unmarshal benchmark	Library	Speed	Allocs/op	Bytes/op
Small (80 B)	encoding/json	9.7 MB/s	14	720
Small (80 B)	easyjson	67 MB/s	3	128
Regular (13 kB)	encoding/json	22 MB/s	218	10,229
Regular (13 kB)	easyjson	125 MB/s	128	9,794

At 100k RPS, 14 allocations per request means 1.4 million allocations per second, and 218 allocations per request means about 21.8 million. The table also shows that the regular-sized document still costs easyjson 128 allocations. Code generation reduces allocations but does not eliminate them, which is why the string-handling options below matter.

2. Verify First, Parse Second

Every major webhook provider signs deliveries with an HMAC, so you can reject bad traffic before any JSON is parsed. The schemes differ in detail:

• GitHub sends X-Hub-Signature-256, formatted sha256=<hex>, computed as HMAC-SHA256 over the raw body. GitHub’s documentation recommends a constant-time comparison. The header is absent if no secret was configured on the webhook.
• Stripe sends a Stripe-Signature header containing a timestamp (t=) and one or more v1= signatures. The signed string is the timestamp, a literal ., and the raw body. A five-minute timestamp tolerance is the common default for rejecting replays.
• Shopify sends X-Shopify-Hmac-Sha256, an HMAC-SHA256 of the raw body encoded as base64 rather than hex.

Two rules follow from this. Always compute the HMAC over the exact bytes received, never over a re-serialized object, because re-encoding changes the bytes and breaks the signature. And always compare signatures in constant time. In Rust, the hmac crate’s verify_slice does this. In Go, use hmac.Equal.

3. Rust: Borrowing from the Request Buffer

Serde supports zero-copy deserialization. If a struct field is &'a str, serde_json can return a slice that points directly into the input buffer instead of allocating a String.

There is one important catch. A string can only be borrowed if it contains no escape sequences. If a JSON string contains something like \n or \", the decoder must rewrite it, so it cannot hand back a pointer into the original bytes. With a bare &str field, deserialization then fails with an error. Serde cannot detect this at compile time, so the failure appears at runtime on real traffic.

For untrusted input such as webhooks, use Cow<'a, str> with #[serde(borrow)] instead. It borrows when it can and falls back to an owned String only when unescaping is required. Both the Cow type and the attribute are needed, because serde defaults to the owned variant otherwise. (#[serde(borrow)] is implicit for &str and &[u8] fields, but not for Cow.)

This also means axum’s standard Json extractor is not the right tool for borrowed types, because it requires owned data. Take the body as Bytes and call serde_json::from_slice yourself, or use the JsonDeserializer extractor from axum-extra, which exists specifically for zero-copy deserialization.

The payload below is illustrative. Real GitHub, Stripe and Shopify events are larger and nested.

use std::{borrow::Cow, sync::Arc};

use axum::{
    body::Bytes,
    extract::{DefaultBodyLimit, State},
    http::{HeaderMap, StatusCode},
    routing::post,
    Router,
};
use hmac::{Hmac, Mac};
use serde::Deserialize;
use sha2::Sha256;
use tokio::sync::mpsc;

type HmacSha256 = Hmac<Sha256>;

#[derive(Debug, Deserialize)]
struct WebhookPayload<'a> {
    #[serde(borrow)]
    event_id: Cow<'a, str>,
    #[serde(borrow)]
    action: Cow<'a, str>,
    timestamp: u64,
    #[serde(borrow)]
    actor: Cow<'a, str>,
}

#[derive(Clone)]
struct AppState {
    secret: Arc<[u8]>,
    queue: mpsc::Sender<Bytes>,
}

fn verify_github(secret: &[u8], header: &str, body: &[u8]) -> bool {
    let Some(hex_sig) = header.strip_prefix("sha256=") else { return false };
    let Ok(sig) = hex::decode(hex_sig) else { return false };
    let Ok(mut mac) = HmacSha256::new_from_slice(secret) else { return false };
    mac.update(body);
    mac.verify_slice(&sig).is_ok() // constant-time comparison
}

async fn ingest_webhook(
    State(state): State<AppState>,
    headers: HeaderMap,
    body: Bytes,
) -> StatusCode {
    // 1. Authenticate the raw bytes before parsing anything.
    let Some(sig) = headers
        .get("x-hub-signature-256")
        .and_then(|v| v.to_str().ok())
    else {
        return StatusCode::UNAUTHORIZED;
    };
    if !verify_github(&state.secret, sig, &body) {
        return StatusCode::UNAUTHORIZED;
    }

    // 2. Hand off a cheap, reference-counted handle to the same buffer.
    //    Bytes::clone() bumps a refcount; it does not copy the payload.
    match state.queue.try_send(body) {
        Ok(()) => StatusCode::ACCEPTED,
        Err(_) => StatusCode::SERVICE_UNAVAILABLE, // queue full: apply backpressure
    }
}

async fn worker(mut rx: mpsc::Receiver<Bytes>) {
    while let Some(body) = rx.recv().await {
        // The parsed struct borrows from `body` and lives only inside this scope.
        if let Ok(event) = serde_json::from_slice::<WebhookPayload>(&body) {
            tracing::info!(id = %event.event_id, action = %event.action, "event received");
        }
    }
}

#[tokio::main]
async fn main() {
    let (tx, rx) = mpsc::channel::<Bytes>(10_000);
    tokio::spawn(worker(rx));

    let state = AppState {
        secret: Arc::from(b"replace-with-your-webhook-secret".as_slice()),
        queue: tx,
    };

    let app = Router::new()
        .route("/webhook", post(ingest_webhook))
        .layer(DefaultBodyLimit::max(1024 * 1024)) // cap request bodies at 1 MiB
        .with_state(state);

    let listener = tokio::net::TcpListener::bind("0.0.0.0:8080").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}

A few details in this code matter:

• Lifetimes. A struct with a borrowed field cannot outlive the buffer it borrows from. Because it cannot be sent across tasks on its own, send the Bytes handle instead. Bytes is a cheaply cloneable, reference-counted buffer, so queueing it does not copy the payload.
• Body limits. Axum’s Bytes extractor refuses bodies larger than 2 MB by default. Set an explicit limit that matches your largest legitimate payload, as shown above.
• Hot-path logging. Avoid println! on the hot path. Use a structured logger such as tracing with sampling.
• Scope of the claim. “Zero-copy” here applies to string fields during parsing. The framework still buffers the body, the HMAC helper here decodes the hex signature into a small Vec, and escaped strings fall back to owned Strings. Verify allocation counts with a profiler such as dhat rather than assuming zero.

4. Go: Generated Parsers with easyjson

encoding/json has traditionally relied on runtime reflection. easyjson avoids that by generating UnmarshalEasyJSON and MarshalEasyJSON methods at build time. Its README claims it outperforms encoding/json by a factor of 4–5x and other JSON packages by 2–3x. It also generates standard MarshalJSON/UnmarshalJSON methods, so types remain compatible with encoding/json, but you only get the speedup by calling easyjson.Unmarshal directly.

Some practical details:

• The generator does not work on package main files, so put your types in a separate package.
• Add //go:generate easyjson -all webhook.go and run go generate. This produces webhook_easyjson.go.
• By default, string fields are copied out of the input buffer, which costs allocations. A field tagged json:"actor,nocopy" makes the string refer to the original buffer memory instead. That is only safe if the decoded object does not outlive the buffer. The README notes this works well for short-lived objects, and strings that need unescaping are processed normally.
• A string field with the intern option is deduplicated, which helps for fields that take only a few distinct values, such as an action name.
• easyjson uses unsafe for no-copy conversions during unmarshaling. You can build without it using the easyjson_nounsafe build tag.
• Its own limitations are documented: object keys are case-sensitive, there is no true streaming, and the parser does only the minimum work needed to skip unmatched values rather than fully validating them.

// webhook.go
package webhook

//go:generate easyjson -all webhook.go

type Payload struct {
    EventID   string `json:"event_id"`
    Action    string `json:"action,intern"`
    Timestamp uint64 `json:"timestamp"`
    Actor     string `json:"actor"`
}

// handler.go
package webhook

import (
    "bytes"
    "crypto/hmac"
    "crypto/sha256"
    "encoding/hex"
    "net/http"
    "strings"
    "sync"

    "github.com/mailru/easyjson"
)

const maxBody = 1 << 20 // 1 MiB

var bufPool = sync.Pool{New: func() any { return new(bytes.Buffer) }}

func validSignature(secret []byte, header string, body []byte) bool {
    const prefix = "sha256="
    if !strings.HasPrefix(header, prefix) {
        return false
    }
    got, err := hex.DecodeString(header[len(prefix):])
    if err != nil {
        return false
    }
    mac := hmac.New(sha256.New, secret)
    mac.Write(body)
    return hmac.Equal(got, mac.Sum(nil)) // constant-time comparison
}

func Handler(secret []byte) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        buf := bufPool.Get().(*bytes.Buffer)
        buf.Reset()
        defer bufPool.Put(buf)

        // Bound the body size and reuse a pooled buffer instead of io.ReadAll.
        if _, err := buf.ReadFrom(http.MaxBytesReader(w, r.Body, maxBody)); err != nil {
            http.Error(w, "payload too large or unreadable", http.StatusRequestEntityTooLarge)
            return
        }
        body := buf.Bytes()

        // Authenticate the raw bytes first.
        if !validSignature(secret, r.Header.Get("X-Hub-Signature-256"), body) {
            http.Error(w, "invalid signature", http.StatusUnauthorized)
            return
        }

        var p Payload
        if err := easyjson.Unmarshal(body, &p); err != nil {
            http.Error(w, "invalid JSON", http.StatusBadRequest)
            return
        }

        // Strings in p are copies (no "nocopy" tags), so p is safe to use
        // after the buffer returns to the pool. If you add "nocopy", it is not.
        w.WriteHeader(http.StatusAccepted)
    }
}

The pooled buffer cuts the per-request allocation of the body slice. That is useful, but be careful: the moment you tag a field nocopy, you must finish using the decoded object before the deferred Put runs.

5. What Changed in Go Recently

Two Go releases change the picture for this kind of service:

• Green Tea garbage collector (Go 1.26). It was an experiment in Go 1.25 and is enabled by default in Go 1.26. The release notes say it improves marking and scanning of small objects and expect a 10–40% reduction in GC overhead for programs that lean heavily on the garbage collector. That helps allocation-heavy parsers, but it does not remove the benefit of allocating less.
• encoding/json/v2 (Go 1.27). Go 1.27 was released on August 19, 2026. It makes encoding/json/v2 and encoding/json/jsontext generally available, and encoding/json itself is now backed by the v2 implementation, with the stated goal of faster unmarshaling. The v2 API has stricter defaults than v1, such as rejecting duplicate object keys and invalid UTF-8, so don’t assume switching is behavior-neutral. One independent benchmark found encoding/json/v2 unmarshaling 1.5x to 2.3x faster than the original implementation, with mixed marshaling results depending on the workload.

The practical consequence is that the headline gap between easyjson and the standard library may be smaller today than the 2016-era README numbers suggest. Benchmark easyjson against Go 1.27’s encoding/json and encoding/json/v2 on your actual payloads before adding a code generator to your build.

If you are considering json-iterator/go as a reflection-free alternative, note that it is not actually reflection-free. It builds and caches per-type decoders using reflection (through reflect2), and its repository was archived on December 15, 2025. For a new service, easyjson or the standard library are safer choices.

6. Measure It Yourself

Treat any numbers in this article, including the ones above, as starting points. To get figures that apply to your service:

1. Use a realistic payload. Benchmark with real, anonymized webhook bodies from the providers you integrate with, not a synthetic struct.
2. Count allocations. In Go, run go test -bench . -benchmem and watch allocs/op. In Rust, use a profiler like dhat or a counting allocator in tests.
3. Load test end to end. Parsing is one stage. Network I/O, TLS, signature verification, queue writes and storage often dominate once parsing is fast.
4. Watch tail latency. Compare p99 latency and GC pause or allocator behavior under sustained load, not only averages.

Key Takeaways

• Authenticate raw bytes before parsing. It is cheaper, safer, and the only way to get the signature right, since all three major providers sign the exact body they send.
• In Rust, use Cow<'a, str> with #[serde(borrow)] for untrusted JSON. Bare &str fields fail at runtime on any escaped string.
• Enqueue Bytes, not parsed structs. A reference-counted buffer handle crosses task boundaries cheaply.
• In Go, easyjson reduces allocations but does not remove them. Its nocopy and intern tags cut them further, at the price of buffer-lifetime discipline.
• Re-benchmark on current Go. Go 1.26’s default GC and Go 1.27’s v2-backed encoding/json both change the baseline.
• Set explicit body-size limits in both languages, and measure with your own traffic.

References

• easyjson README and benchmarks
• GitHub: Validating webhook deliveries
• axum DefaultBodyLimit
• axum-extra JsonDeserializer (zero-copy JSON extractor)
• serde’s borrowing can be treacherous
• Go 1.26 release notes (Green Tea GC)
• Go 1.27 release announcement
• Go’s experimental JSON v2 announcement
• The new Go JSON API: twice as fast, or 1.5x slower?

---

Originally published at https://instawebhook.com/blog/zero-copy-webhook-parsing-optimizing-rust-and-go-ingress-pipelines-for-100k-rps