Multi-Server-and-Process-Management

Multi-Server and Process Management in Hyperlane

Project Code：https://github.com/hyperlane-dev/hyperlane

Introduction

One of Hyperlane's strengths is its ability to run multiple HTTP server instances concurrently within a single process. This capability is essential for modern web applications that need to serve different APIs on different ports, run administrative endpoints alongside public APIs, or isolate services for security and reliability. This article explores Hyperlane's multi-server architecture, process management patterns, and graceful shutdown mechanisms.

Why Multiple Servers?

Running multiple servers in a single process offers several advantages:

1. Port separation — Serve different services on different ports (e.g., API on port 80, admin on port 81)
2. Protocol isolation — Run HTTP and WebSocket services independently
3. Security boundaries — Isolate public and internal endpoints
4. Resource management — Apply different configurations to different services
5. Zero-downtime updates — Bring up new instances before shutting down old ones

Basic Multi-Server Setup

Hyperlane makes it straightforward to run multiple servers using Tokio's task spawning:

let app1 = tokio::spawn(async move {
    let mut sc = ServerConfig::default();
    sc.set_address(Server::format_bind_address(DEFAULT_HOST, 80));
    let mut s = Server::default();
    s.server_config(sc);
    s.run().await.unwrap_or_default().wait().await;
});

let app2 = tokio::spawn(async move {
    let mut sc = ServerConfig::default();
    sc.set_address(Server::format_bind_address(DEFAULT_HOST, 81));
    let mut s = Server::default();
    s.server_config(sc);
    s.run().await.unwrap_or_default().wait().await;
});

let _ = tokio::join!(app1, app2);

In this example, two server instances run concurrently on ports 80 and 81. Each is spawned as an independent Tokio task, and tokio::join! waits for both to complete (which normally happens only on shutdown).

Server Configuration for Multi-Server

Each server instance can have its own ServerConfig:

let mut config: ServerConfig = ServerConfig::default();
config.set_address("0.0.0.0:80");
config.set_nodelay(Some(true));
config.set_ttl(Some(128));

When creating multiple servers, ensure each one uses a unique address. The Server::format_bind_address helper makes this convenient:

sc.set_address(Server::format_bind_address(DEFAULT_HOST, 80));
sc.set_address(Server::format_bind_address(DEFAULT_HOST, 81));

Per-Server Middleware and Routes

Each server instance is completely independent. You can register different middleware and routes on each:

let app1 = tokio::spawn(async move {
    let mut sc = ServerConfig::default();
    sc.set_address(Server::format_bind_address(DEFAULT_HOST, 80));
    let mut s = Server::default();
    s.server_config(sc);
    s.route::<PublicApi>("/api/public");
    s.run().await.unwrap_or_default().wait().await;
});

let app2 = tokio::spawn(async move {
    let mut sc = ServerConfig::default();
    sc.set_address(Server::format_bind_address(DEFAULT_HOST, 81));
    let mut s = Server::default();
    s.server_config(sc);
    s.route::<AdminApi>("/admin");
    s.request_middleware::<AuthMiddleware>();
    s.run().await.unwrap_or_default().wait().await;
});

In this setup, the public API on port 80 has no authentication, while the admin API on port 81 requires it. Each server has its own middleware stack and route table.

Graceful Shutdown

Hyperlane supports graceful shutdown through the ServerControlHook:

let server_control_hook = server.run().await.unwrap_or_default();
server_control_hook.shutdown().await;

The shutdown() method signals the server to stop accepting new connections and finish processing existing ones before shutting down. This is critical for multi-server setups where you want to shut down all instances cleanly.

For multi-server shutdown, you need to coordinate the shutdown of each server's control hook:

let mut server1 = Server::default();
server1.server_config(sc1);
let hook1 = server1.run().await.unwrap_or_default();

let mut server2 = Server::default();
server2.server_config(sc2);
let hook2 = server2.run().await.unwrap_or_default();

// Later, shutdown both servers
hook1.shutdown().await;
hook2.shutdown().await;

Connection Management Across Servers

Each server manages its own connections independently. The keep-alive and connection management settings are per-server:

stream.set_closed(true);
let keep_alive = stream.is_keep_alive(ctx.get_request().is_enable_keep_alive());
while stream.try_get_http_request().await.is_ok() {
    if !ctx.get_request().is_enable_keep_alive() {
        stream.set_closed(true);
        break;
    }
}

This pattern works identically within each server instance. Connection reuse (keep-alive) is managed per-connection within each server's scope.

Request Configuration Per Server

Each server can have its own RequestConfig for fine-tuning request handling:

let request_config_json = r#"{
    "buffer_size": 8192,
    "max_path_size": 8192,
    "max_header_count": 100,
    "max_header_key_size": 8192,
    "max_header_value_size": 8192,
    "max_body_size": 2097152,
    "read_timeout_ms": 6000
}"#;
let request_config = RequestConfig::from_json(request_config_json).unwrap();
let mut server = Server::from(request_config);

This allows you to set different body size limits, timeouts, and buffer sizes for different servers. For example, a file upload server might need a larger max_body_size than a typical API server.

JSON Configuration for Multi-Server

Hyperlane supports configuring servers from JSON, which is particularly useful for multi-server setups where you want to manage configurations externally:

let config_json = r#"{ "address": "0.0.0.0:80", "nodelay": true, "ttl": 64 }"#;
let mut server = Server::default();
server.config_from_json(config_json);

You can load different JSON configurations for each server instance from files, environment variables, or a configuration service.

Using the Hyperlane Attribute Macro for Multi-Server

Hyperlane's attribute macro can simplify server setup:

#[hyperlane(server: Server)]
#[hyperlane(server_config: ServerConfig)]
#[tokio::main]
async fn main() {
    server.server_config(server_config);
    let server_control_hook = server.run().await.unwrap_or_default();
    server_control_hook.wait().await;
}

While this pattern is typically used for single-server setups, you can still leverage it as part of a multi-server architecture by spawning tasks that each use this macro.

Error Handling in Multi-Server Setups

Each server has its own error handling middleware:

struct RequestErrorHook;

impl ServerHook for RequestErrorHook {
    async fn new(_: &mut Stream, ctx: &mut Context) -> Self {
        let request_error = ctx.try_get_request_error_data().unwrap_or_default();
        Self
    }

    async fn handle(self, stream: &mut Stream, ctx: &mut Context) -> Status {
        let data = ctx.get_mut_response().set_status_code(500).set_body("error").build();
        stream.try_send(data).await;
        Status::Continue
    }
}

server.request_error::<RequestErrorHook>();

Register error hooks on each server independently. This ensures that errors on one server don't affect the error handling of another.

Task Panic Handling

Similarly, each server can have its own panic handler:

struct TaskPanicHook;

impl ServerHook for TaskPanicHook {
    async fn new(_: &mut Stream, ctx: &mut Context) -> Self {
        let error = ctx.try_get_task_panic_data().unwrap_or_default();
        Self
    }

    async fn handle(self, stream: &mut Stream, ctx: &mut Context) -> Status {
        let data = ctx.get_mut_response().set_status_code(500).set_body("panic").build();
        stream.try_send(data).await;
        Status::Continue
    }
}

server.task_panic::<TaskPanicHook>();

This is especially important in multi-server setups because a panic in one server task should not crash the other server instances.

Performance Considerations

When running multiple servers, keep these performance tips in mind:

1. Tokio runtime — All servers share the same Tokio runtime. Ensure the runtime has enough threads for all servers.
2. Resource limits — Each server maintains its own connection pool. Monitor total file descriptor usage.
3. CPU affinity — For high-throughput scenarios, consider pinning server tasks to specific CPU cores.
4. Load balancing — For identical server instances behind a load balancer, ensure consistent configuration.

Hyperlane's performance benchmarks demonstrate its efficiency:

• Without Keep-Alive: hyperlane QPS 51031, Tokio 49555, Rocket 49345, Gin 40149
• With Keep-Alive: Tokio 340130, hyperlane 334888, Rocket 298945, Gin 242570
• ab test with 1 million requests: hyperlane 316211 QPS (Keep-Alive), Tokio 308596

Best Practices

1. Use unique addresses — Never bind two servers to the same address.
2. Register error handlers on each server — Don't assume shared state.
3. Coordinate shutdown — Use ServerControlHook::shutdown() for each server.
4. Monitor independently — Track metrics per server, not just per process.
5. Isolate configurations — Each server should have its own ServerConfig and RequestConfig.
6. Handle panics gracefully — Register TaskPanicHook on every server to prevent cascading failures.

Conclusion

Hyperlane's multi-server support makes it easy to run multiple HTTP server instances concurrently within a single process. With independent configurations, middleware stacks, and route tables, each server operates as a fully autonomous unit while sharing the Tokio runtime. Graceful shutdown, per-server error handling, and panic recovery ensure that your multi-server architecture is robust and production-ready.

---

Project Code：https://github.com/hyperlane-dev/hyperlane