DEV Community

Cover image for Basic Handshake protocol
JONATHAN SIMON
JONATHAN SIMON

Posted on

Basic Handshake protocol

#programming #webdev #rust

I wrote a basic handshake protocol in Rust. Let's break down the entire implementation step-by-step, explaining each component and how it addresses the requirements for a secure handshake protocol.

A. Protocol Design Overview
The protocol implements mutual authentication using pre-shared keys (PSK) with the following message sequence:

  • ClientHello:
    Client ID (variable length)
    Client Nonce (16-byte random value)

  • ServerHello:
    Server Nonce (16-byte random value)
    HMAC(PSK, ClientNonce + ServerNonce + "Server")

  • ClientAuth:
    HMAC(PSK, ClientNonce + ServerNonce + "Client")

  • ServerAck:
    "AUTH_SUCCESS" confirmation

B. Security Features Implemented:

  • Mutual Authentication: Both client and server prove knowledge of PSK
  • Replay Attack Prevention: Unique nonces for each session
  • Message Integrity: HMAC-SHA256 protects against tampering
  • Context Separation: Different HMAC contexts for client/server roles
  • Freshness Guarantee: Nonces ensure responses are current

C. Key Components Explained:

a. KeyStore (protocol.rs)

#[derive(Clone)]
pub struct KeyStore(HashMap<String, Vec<u8>>);
Enter fullscreen mode Exit fullscreen mode
  • Purpose: Stores pre-shared keys for clients
  • Implementation:
    • Uses HashMap for client ID → key mapping
    • #[derive(Clone)] enables safe sharing between threads
    • get_key() retrieves key with error handling

b. Nonce Generation

pub fn generate_nonce() -> [u8; NONCE_SIZE] {
    let mut nonce = [0u8; NONCE_SIZE];
    rand::rngs::OsRng.fill_bytes(&mut nonce);
    nonce
}
Enter fullscreen mode Exit fullscreen mode
  • Security Importance:
    • Prevents replay attacks
    • Ensures session uniqueness
  • Implementation:
    • Uses cryptographically secure OsRng
    • Generates 16 random bytes (128 bits)

c. HMAC Operations

pub fn compute_hmac(key: &[u8], data: &[&[u8]]) -> Result<[u8; 32]> {
    let mut mac = HmacSha256::new_from_slice(key)?;
    for d in data {
        mac.update(d);
    }
    Ok(mac.finalize().into_bytes().into())
}
Enter fullscreen mode Exit fullscreen mode
  • Security Properties:
    • Keyed-Hash Message Authentication Code
    • SHA-256 provides collision resistance
  • Implementation Notes:
    • Concatenates inputs: [c_nonce, s_nonce, role]
    • Role strings ("Client"/"Server") prevent reflection attacks

D. Server Implementation (server.rs)
a. Initialization

let listener = TcpListener::bind("127.0.0.1:8080").await?;
let keystore = Arc::new(KeyStore::new());
Enter fullscreen mode Exit fullscreen mode
  • Concurrency Handling:
    • Arc enables thread-safe reference counting
    • Allows sharing keystore across connections

b. Connection Handling

tokio::spawn(async move {
    // Handshake logic
});
Enter fullscreen mode Exit fullscreen mode
  • Asynchronous Design:
    • Uses Tokio runtime for async I/O
    • Spawns new task per connection

c. Handshake Sequence
i. Receive ClientHello:

let client_id_len = socket.read_u8().await?;
socket.read_exact(&mut client_id_buf[..client_id_len]).await?;
socket.read_exact(&mut c_nonce).await?;
Enter fullscreen mode Exit fullscreen mode
  • Reads client ID (length-prefixed)
  • Reads 16-byte client nonce

ii. Send ServerHello:

let server_hmac = compute_hmac(psk, &[&c_nonce, &s_nonce, b"Server"])?;
socket.write_all(&s_nonce).await?;
socket.write_all(&server_hmac).await?;
Enter fullscreen mode Exit fullscreen mode
  • Proves server knows PSK
  • Binds response to client's nonce

iii. Verify ClientAuth:

verify_hmac(psk, &[&c_nonce, &s_nonce, b"Client"], &client_hmac)?;
Enter fullscreen mode Exit fullscreen mode
  • Validates client knows PSK
  • Ensures client received server's nonce

iv. Send Acknowledgement:

socket.write_all(b"AUTH_SUCCESS").await?;
Enter fullscreen mode Exit fullscreen mode
  • 12-byte fixed message confirms auth

E. Client Implementation (client.rs)
a. Handshake Sequence
i. Send ClientHello:

socket.write_u8(client_id.len() as u8).await?;
socket.write_all(client_id.as_bytes()).await?;
socket.write_all(&c_nonce).await?;
Enter fullscreen mode Exit fullscreen mode
  • Length-prefixed client ID
  • Random client nonce

ii. Verify ServerHello:

verify_hmac(psk, &[&c_nonce, &s_nonce, b"Server"], &server_hmac)?;
Enter fullscreen mode Exit fullscreen mode
  • Validates server identity
  • Ensures response is fresh (contains client nonce)

iii. Send ClientAuth:

let client_hmac = compute_hmac(psk, &[&c_nonce, &s_nonce, b"Client"])?;
socket.write_all(&client_hmac).await?;
Enter fullscreen mode Exit fullscreen mode
  • Proves client knows PSK
  • Binds response to server's nonce

iv. Verify ServerAck:

if &ack == b"AUTH_SUCCESS" {
    println!("Authenticated with server!");
}
Enter fullscreen mode Exit fullscreen mode
  • Final confirmation of successful auth

F. Protocol Security Analysis
a. Replay Attack Prevention

  • Client Nonce: Ensures server response is fresh
  • Server Nonce: Ensures client authentication is fresh
  • HMAC Context: Includes both nonces in all authentication tags

b. Authentication Strength

  • Mutual Authentication: Both parties prove PSK knowledge
  • Context Separation: Different HMAC strings for client/server roles
  • Key Binding: All HMACs incorporate both parties' nonces

c. Message Sequencing

  • Strict state machine enforced by:
    • Server waits for ClientHello before responding
    • Client waits for ServerHello before authenticating
    • Server waits for ClientAuth before acking

d. Minimal Security Concepts

  • Authentication: HMAC proves PSK knowledge
  • Integrity: HMAC protects against message tampering
  • Freshness: Nonces guarantee message recency
  • Key Confidentiality: PSK never transmitted

This implementation provides a solid foundation for secure protocol design, demonstrating core concepts like mutual authentication, nonce-based freshness, and HMAC-based message integrity. The Rust implementation leverages async I/O for performance and type safety for correctness.

Top comments (0)