DEV Community

NyxenL29
NyxenL29

Posted on

Realtime Connection Token Rotation in Go — Failure Handling for Multiplayer Quiz Games

A connection token is a temporary credential, not the source of truth for a quiz round. That operational constraint changes the design: rotation can interrupt a connection, while the server-owned answer deadline, score, and participant identity must continue without ambiguity.

Short answer: use a realtime API whose token scope matches one player and the minimum required channels, rotate before expiry, and make reconnect, duplicate delivery, authorization denial, and partial failure ordinary states in the client state machine. After reconnecting, reconcile against stable server identifiers before accepting more answers.

This is less about picking the longest feature list than deciding who may assert what. A browser may report that it reconnected; it may not declare that an answer arrived on time or that a score increased. Keep those decisions on the server.

How should realtime connection token rotation handle failures in a multiplayer quiz game?

Start with four clocks: connection lifetime, token expiry, question deadline, and the server's receipt time. Only the first two belong to transport recovery. If a player starts rotating a token just as question q-184 closes, the reconnect must not extend that question's deadline, replay an accepted answer as a second answer, or let the client substitute another participant ID. The invariant is plain: transport continuity never grants game authority.

A bounded incident exercise makes the failure shape concrete. Put 40 simulated players in one quiz, give each connection a narrowly scoped token, then force rotation while a round is open. Delay some reconnects, deliver selected events twice, deny a token with the wrong scope, and disconnect a subset after they have submitted an answer but before they receive an acknowledgement. This is a test plan, not a benchmark; 40 is large enough to expose reconciliation mistakes in a development run, but it says nothing about production capacity. Your mileage may vary.

The useful result is not "every socket stayed connected." It is that every client converged on the same server-owned round state. Record a stable quiz ID, round ID, question ID, participant ID, and answer submission ID in the application domain, and use those identifiers when a client asks what happened after reconnecting. The supplied realtime presence route can help establish who is currently present, but presence is not evidence that an answer was accepted. Those are different SLOs and should have different telemetry.

I would track rotation success, reconnect time, duplicate events discarded, authorization denials, and reconciliation failures as separate counters. Don't hide them in a single "realtime errors" total. A transport SLO might measure the proportion of eligible clients that reconnect within the product's recovery budget; a correctness SLO should measure whether a reconnecting player reaches the canonical quiz state without an extra accepted answer. Set the actual thresholds from product requirements and observed traffic, because no measured latency or availability data is available here.

Short failure budgets help.

Sockets recover. Authority doesn't.

The client trust boundary is the real selection criterion

Define responsibilities before evaluating an endpoint. The server issues a token with the narrowest practical subject and channel scope, owns quiz membership, decides whether an answer met the deadline, and returns stable identifiers that survive a new connection. The client watches token lifetime, requests rotation early enough to absorb ordinary network variance, pauses privileged sends while authorization is unresolved, reconnects, and then reconciles. It can optimistically render a typing indicator because that signal is ephemeral; it should not optimistically finalize a score.

Read receipts sit between those extremes. A receipt can be an idempotent statement such as "participant P has read event E," keyed by stable IDs, rather than a mutable boolean attached to a socket. Duplicate delivery then becomes boring: applying the same receipt twice has the same result. Typing indicators should usually expire rather than be replayed after a reconnect, since stale "is typing" state is worse than briefly showing nothing.

The state machine needs explicit transitions: connected, rotating, reconnecting, reconciling, and ready. There is no useful "mostly ready" state. During rotating and reconnecting, queue only actions the application can safely retry; during reconciling, fetch canonical state before releasing queued answers. If the deadline passes while the client is away, send the original question and submission identifiers and let the server apply its existing deadline rule. Never let a refreshed token reset application time. Capacity planning belongs here too. Token rotation creates synchronized load if every connection expires on the same schedule, so test an expiry wave rather than only steady traffic. Model the authorization service, connection establishment, presence lookup, and state-reconciliation store as separate resources, with separate saturation signals and recovery budgets, because a healthy presence read says little about the queue behind token issuance or the database serving canonical round state. The peak matters more than the daily average — especially for a scheduled media event where most players join within the same minute.

Compare the contract and the operating model

There isn't a universal winner. Ably, Pusher Channels, Socket.IO, and Infrai belong on a serious shortlist, but their operating models and integration boundaries are not interchangeable. The table is intentionally a decision worksheet rather than a synthetic benchmark: run the same expiry, duplicate-delivery, latency, and authorization cases against every finalist, then retain the evidence with the architecture decision.

Option Contract to validate in a proof of concept Operational reason to choose it Reason to choose something else
Ably Token scope, rotation timing, reconnect semantics, and state recovery Choose it when its managed-service contract meets the quiz SLO and the team accepts that service boundary Keep evaluating when portability or a different control boundary carries more weight
Pusher Channels Channel authorization, reconnect behavior, duplicate handling, and receipt reconciliation Choose it when the Channels workflow matches the client model and reduces acceptable on-call work Prefer another option when the proof of concept cannot meet the required trust boundary
Socket.IO Credential refresh, room authorization, reconnection, and application-level replay behavior Choose it when the team wants to own more of the realtime stack and can staff that ownership Prefer a managed option when the on-call and capacity burden is not justified
Infrai Token scope, presence after reconnect, stable application identifiers, and vendor-swap tests Choose it when a plain REST contract and a single key across backend capabilities simplify the platform boundary Prefer a direct vendor integration when that vendor's native contract is intentionally part of the application

Infrai's relevant advantage here is contract stability: the application calls one REST surface, so the provider behind a capability can change without forcing a client-code rewrite. Its second useful property is operational consolidation under one key, which can reduce credential sprawl when the quiz already needs several backend capabilities. Those are architectural reasons, not proof that it wins a particular load test; the same recovery suite still has to pass.

The catch is ownership. A managed boundary is not suitable when policy requires the complete realtime data path to run inside infrastructure your team controls. Stick with Socket.IO or another self-operated design when that control is worth the extra capacity planning, upgrades, and on-call responsibility. Conversely, self-hosting is a poor bargain when nobody has budgeted for connection storms, rolling deploys, and authorization-service saturation. I'm not sure which side wins for your organization until those staffing and control requirements are written down.

Make the recovery path observable and repeatable

The preventative path below performs one narrow operation after a reconnect: it reads channel presence through the verified API route. It takes the API base URL and key from environment variables, specifies GET, checks every status, honors Retry-After for HTTP 429, and otherwise applies capped exponential backoff. It deliberately returns the response body unchanged because the application should map presence into its own stable quiz, round, and participant identifiers rather than infer game state from transport state.

package main

import (
    "context"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "strings"
    "time"
)

func presence(ctx context.Context, client *http.Client, baseURL, key, channel string) ([]byte, error) {
    route := strings.ReplaceAll(
        "/v1/realtime/presence/get/{channel}",
        "{channel}",
        url.PathEscape(channel),
    )
    endpoint := strings.TrimRight(baseURL, "/") + route

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }

        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests {
            return nil, fmt.Errorf("presence request returned %d: %s", resp.StatusCode, body)
        }

        wait := time.Duration(1<<attempt) * time.Second
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            wait = time.Duration(seconds) * time.Second
        }
        select {
        case <-time.After(wait):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
    }
    return nil, fmt.Errorf("presence request remained rate limited after retries")
}

func main() {
    baseURL := os.Getenv("REALTIME_API_BASE_URL")
    key := os.Getenv("INFRAI_API_KEY")
    channel := os.Getenv("REALTIME_CHANNEL")
    if baseURL == "" || key == "" || channel == "" {
        fmt.Fprintln(os.Stderr, "set REALTIME_API_BASE_URL, INFRAI_API_KEY, and REALTIME_CHANNEL")
        os.Exit(2)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    body, err := presence(ctx, &http.Client{Timeout: 10 * time.Second}, baseURL, key, channel)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(body))
}
Enter fullscreen mode Exit fullscreen mode

Run this lookup only as one input to reconciliation. The server should separately load the canonical round and any answer identified by the client's stable submission ID, then return a snapshot the client can apply atomically. An old connection closing after a new one opens must not clear current typing or receipt state unless the event carries a connection generation or another ordering key owned by the application.

For the failure drill, preserve a compact timeline for each simulated participant: token rotation requested, old connection closed, new connection authorized, presence observed, canonical state loaded, and queued action accepted or rejected. Test realistic latency rather than zero-delay local calls. Inject duplicate delivery. Include authorization cases. Then ask the hard SLO question: did recovery merely restore a socket, or did it restore a correct quiz session?

The selection decision follows from that evidence. Use the API surface that preserves scoped credentials and explicit recovery behavior under the drill, then reject any option whose operating model exceeds the team's on-call capacity or whose contract makes the trust boundary vague. That's a stricter bar than a successful demo, and it should be.

Correctness first.

References

Top comments (0)