NielsChristensen4981

Posted on Oct 12

Realtime Publish: 5 Batch Versus Individual Choices for Simultaneous Support Updates

#realtime #security #sre

Short answer: batch simultaneous realtime updates when they share an authorization scope, ordering boundary, and failure policy; publish them individually when any of those differ. For a live customer-support dashboard, the useful unit is usually one tenant-scoped snapshot or delta bundle, not one giant cross-tenant payload and not one network publish per ticket. Keep authorization on the server, give the browser a short-lived capability for one tenant and purpose, attach a monotonic sequence to every bundle, and let the client reject gaps rather than trust whatever arrives.

The page fires at 02:14: support_dashboard_freshness has missed its service-level objective. On-call sees connected browsers, a healthy publish endpoint, and agents reporting that assignment badges lag behind ticket state. The tempting response is to raise the alert threshold because the transport looks fine. That would hide the signal. Work backward: the page should have been preceded by rising end-to-end apply latency and sequence gaps, which distinguish a delivery-looking-healthy incident from a dashboard that is actually current.

This is why “batch or individual?” is the wrong first question. The first question is what one client is allowed to learn and what one failed operation is allowed to delay.

1. Why did a healthy connection deliver a stale dashboard?

A connection metric proves very little about correctness. A browser can remain connected while updates wait in a server queue, arrive out of order after retries, or get discarded because the local view has moved past the update's assumed state. For support work, that difference matters: an agent looking at ticket T-1842 needs to know whether its assignment is current, not merely whether bytes crossed a socket.

Connected can still mean stale.

Define the user-facing objective first. A defensible freshness SLO might say that an authorized assignment change is visible on the intended tenant dashboard within a chosen latency target, measured from committed source state to successful client application. The target itself must come from product risk and observed workload; there is no universal number in the supplied standards. Measure all four timestamps: commit, enqueue, receive, and apply. Then page on the end-to-end result, while using the intermediate spans to locate the delay.

The earlier warning signal is a widening difference between the highest committed sequence and the highest applied sequence, grouped by tenant and dashboard cohort. Queue depth is supporting evidence, not the objective. Connection count is capacity data, not success.

That distinction changes the publishing choice. Individual publishes make per-event failures visible and isolate retries, but they multiply scheduling, serialization, and acknowledgement work during a burst. A batch amortizes that work and can preserve a coherent view, but one oversized or unauthorized member can hold up everything behind it. The trade-off isn't merely throughput: larger bundles reduce publish operations while increasing shared fate, memory held per flush, and the amount of work repeated after rejection. Batch only inside the smallest boundary that is already atomic for policy and recovery.

2. Should You Batch Many Simultaneous Realtime Updates or Use Individual Publish Calls?

Partition by authorization before batching. Build a separate bundle for each tenant and permission class. Never create a cross-tenant batch and expect the browser to filter it. The client is an enforcement target, not an enforcement point; developer tools, extensions, logs, and compromised code can expose every field delivered to it.
Choose a consistency unit, not an arbitrary event count. Ticket assignment, queue count, and agent availability may need to advance together if the UI derives one decision from all three. Put those changes under one sequence boundary. Unrelated typing indicators or ephemeral presence can use a separate stream because losing one should not block durable work state.
Cap time and size independently. A batch should flush when either its short collection window expires or its encoded-size ceiling is reached. An item-count-only cap is weak capacity planning: ten small identifiers and ten verbose note previews do not consume the same memory or network budget. Derive both ceilings from load tests, transport limits, and the freshness SLO rather than copying a fashionable value.
Make gaps recoverable. Give every tenant stream a monotonically increasing sequence and make the bundle identify its first and last sequence. A client that receives 1042-1048 after applying 1040 has evidence of a gap; it should request an authorized snapshot or replay instead of guessing that 1041 was unimportant. Retries also require stable event identifiers so repeated delivery does not repeat a side effect.
Separate durable state from lossy hints. Assignment and status changes belong in a recoverable stream. Cursor motion, “agent is viewing,” or typing state can often tolerate replacement by a newer value. Combining both classes makes the durable path inherit needless volume, while the hint path inherits a retention and replay obligation it does not need.

These rules work in a Node.js publisher even though the example below is Go, because they define the envelope and trust boundary rather than a runtime-specific client library. The batching loop is an implementation detail; the security partition is the design.

3. A scoped envelope the browser can distrust safely

The server should derive tenant and allowed topics from authenticated session state. Do not accept a browser-supplied tenant identifier as proof of membership, and do not place broad backend credentials in frontend code. A short-lived client capability should identify its audience, expiry, tenant, and narrowly allowed purpose; the realtime gateway must validate those claims before subscription or replay. If a system uses JSON Web Tokens, RFC 7519 defines standard claims such as aud and exp, but the application still has to define and enforce the meaning of its private scope claims.

Here is a deliberately small envelope. It does not contain credentials, and it gives the receiver enough information to detect duplicates and gaps.

package realtime

import (
    "encoding/json"
    "errors"
)

type Update struct {
    ID       string          `json:"id"`
    Kind     string          `json:"kind"`
    Sequence uint64          `json:"sequence"`
    Payload  json.RawMessage `json:"payload"`
}

type Bundle struct {
    Tenant        string   `json:"tenant"`
    Purpose       string   `json:"purpose"`
    FirstSequence uint64   `json:"first_sequence"`
    LastSequence  uint64   `json:"last_sequence"`
    Updates       []Update `json:"updates"`
}

func (b Bundle) Validate(expectedTenant string, nextSequence uint64) error {
    if b.Tenant != expectedTenant || b.Purpose != "support-dashboard" {
        return errors.New("bundle is outside the authorized scope")
    }
    if len(b.Updates) == 0 || b.FirstSequence != nextSequence {
        return errors.New("empty bundle or sequence gap")
    }
    if b.LastSequence < b.FirstSequence {
        return errors.New("invalid sequence range")
    }
    return nil
}

Validation needs another layer in production: bound the decoded size before allocation, restrict accepted update kinds, verify that each member sequence belongs to the declared range, and handle integer exhaustion according to the storage design. Those checks are easy to miss when a batch is treated as “just an array.” It is a protocol message.

Transport does not remove this responsibility. RFC 6455 defines the WebSocket protocol, while WebRTC defines peer connections and data channels; neither standard decides which support agent may read which ticket. The application owns authorization and recovery semantics in either case.

4. Measure the decision with a buy-versus-build ledger

The operational choice is broader than a request-count comparison. A managed service and a self-hosted gateway can expose similar publish primitives while moving very different duties onto the platform team. I would put the decision in a ledger before permitting a proof of concept to become architecture.

Concern	Managed transport	Self-hosted transport	Evidence to collect
Token enforcement	Confirm claim validation, revocation behavior, and channel isolation	Implement and audit validation at every entry point	Negative authorization tests and audit events
Batch limits	Documented service ceilings and rejection behavior	Limits chosen and enforced by the team	Encoded bytes, members, and rejection rate
Recovery	Available replay or snapshot integration may constrain the design	Team owns durable log, cursor, and compaction	Gap-recovery time and replay correctness
On-call load	Provider owns part of the transport; integration remains yours	Team owns connection fleet, upgrades, and capacity	Pages, toil, saturation, and restore time
Lock-in	Proprietary channel, token, or replay semantics can raise exit cost	Internal protocol can still become accidental lock-in	A tested portable envelope and migration exercise

This table does not select a winner. It prevents request throughput from crowding out on-call cost and exit cost. The capacity model should include peak committed updates per second, fan-out per update, encoded bytes, connections per authorization cohort, retry amplification, and snapshot traffic during recovery. Average traffic is nearly useless for a simultaneous burst.

Bounded batching is not suitable when a low-volume stream gains no measured headroom, when events have different authorization scopes, or when one event must be acknowledged and retried independently. In those cases, individual publish calls are easier to reason about. The downside is more per-event overhead; accepting it is rational when isolation is the higher-order requirement.

Test the same trace in two modes: one publish per durable update, then tenant-scoped bundles with bounded size and delay. Compare end-to-end apply latency percentiles, publisher CPU, allocation pressure, queue age, dropped connections, retry volume, and recovery time after an injected sequence gap. The decision rule is plain: use batching only if it creates headroom without violating the freshness objective or enlarging the trust boundary.

5. Instrument the alert before tuning the threshold

Return to the 02:14 page. Add a trace identifier and tenant-safe correlation key at commit, carry them through enqueue and delivery, and record the client apply acknowledgement against the sequence range. Metrics should aggregate without putting ticket text, customer identifiers, or bearer tokens into labels. Logs need rejection reasons such as expired capability, wrong audience, sequence gap, oversize bundle, and unsupported kind; they do not need the credential itself.

Deploy the envelope reader before the writer. First teach clients to accept the new version and preserve the individual-event path, then canary the bundler for a limited tenant cohort, watch both correctness and saturation, and retain a server-side switch that changes publish mode without broadening token scope. Contract tests should cover reordered bundles, duplicate delivery, partial serialization failure, expiration during reconnect, tenant mismatch, and a snapshot racing with live updates.

Alert on the symptom the user experiences: committed-to-applied freshness breaching its SLO across a meaningful cohort. Use sequence-gap rate, oldest queue age, authorization rejects, and bundle rejection rate as diagnostic alerts with carefully chosen routing. A threshold that is too tight turns harmless short bursts into pages and teaches on-call to distrust the system; one that is too loose lets agents make decisions from stale assignments. The false-positive cost is not abstract. It spends attention, and eventually it suppresses the response to the page that matters.

False positives consume the same on-call budget as real failures.

The final choice is conditional: individual publishing is the safer default when updates cross policy or failure boundaries; bounded batching is the capacity tool when updates already share those boundaries and the receiver can prove continuity. Optimize after that line is drawn.

DEV Community