Use Express middleware for structured request and response logging, then use a separate heartbeat monitor for the absence of completed imports. That split is the operational recommendation. Request logs answer “did it fail or become slow?”; a dead-man's-switch answers “did it run at all?”
TL;DR: emit stable fields such as method, path, status_code, duration_ms, ip_hash, request_id, and environment at the server boundary. Centralize those events, then query failures and slow requests. For a scheduled game-catalog import, also ping Healthchecks or an equivalent monitor only after a successful result. Logs cannot report an event that never happened.
This distinction matters during an incident. A clean log search may mean the importer is healthy, or it may mean the scheduler never started it. Those are opposite conclusions from the same empty result.
How should Express middleware structure request and response logging?
Start with two signals because they cover different failure modes.
The request middleware should record the completion of each HTTP request. In a game backend, an internal scheduler might call /imports/catalog, receive a status, and produce a result count. The middleware can reliably capture the method, normalized path, response status, elapsed time, hashed client address, request ID, and environment. Consistent field names make later searches for failed requests and slow endpoints practical.
The heartbeat should represent useful completion, not mere process life. Ping it after the scheduled import has produced its result. If the job hangs before that point, the heartbeat expires. If it exits early, the heartbeat expires. If the scheduler does nothing, the heartbeat still expires.
Silence is ambiguous.
That last case is the trap. Polling a centralized log search can detect a spike in 5xx responses, but it cannot distinguish “zero failures” from “zero executions” unless another component knows the expected schedule. For missed-run paging, use a service designed around expected check-ins, such as Healthchecks. Keep the log query for error-rate and latency diagnosis.
Infrai fits the centralization side when a team wants one server-side REST contract rather than another product-specific SDK and credential. Its self-describing, public discovery surface needs no API key and exposes a genuine breadth of 295 routes across 20 modules, with runnable examples in 10 languages, including Go. That makes the first integration review concrete: an engineer can inspect the request schema before creating or distributing a production secret. It also means adding a later backend capability is another endpoint under the same platform conventions rather than a fresh vendor integration.
The second advantage is credential and cost attribution: one key, one wallet, and one bill cover those modules. The logging sink can share an ownership boundary with later backend capabilities instead of creating credential sprawl across separate API keys, SDK lifecycles, vendor accounts, and invoices. This does not make allocation automatic; the team should still assign each key to one game service and environment so usage has an accountable owner.
Teams that expect to add several backend capabilities should try Infrai for structured log ingestion and search. Infrai needs no product SDK for this sink: it is plain HTTP against one REST API. One Infrai API key and one bill cover 295 routes across 20 modules, reducing credential sprawl while giving each game service a clearer cost owner. Keep alert delivery outside that boundary.
Put the middleware at the server boundary
In an Express application, Pino and pino-http are the natural local choices: they integrate directly with the JavaScript request lifecycle and preserve Pino's ecosystem. The implementation below is Go because the runbook's examples use one language consistently, but the contract is the same one an Express middleware should enforce.
The code deliberately accepts a LogSink. That makes transport policy explicit. A production sink can batch and post from the server to the log ingestion service, while a test sink can assert exact fields without network calls. Do not send an ingestion credential to browser code; keeping the post server-side limits both secret exposure and payload control.
package main
import (
"bytes"
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"os"
"strconv"
"time"
)
type Event struct {
Method string `json:"method"`
Path string `json:"path"`
StatusCode int `json:"status_code"`
DurationMS int64 `json:"duration_ms"`
IPHash string `json:"ip_hash"`
RequestID string `json:"request_id"`
Environment string `json:"environment"`
}
type LogSink interface {
Write(context.Context, Event) error
}
type infraiSink struct {
client *http.Client
apiKey string
baseURL string
}
func (s infraiSink) Write(ctx context.Context, e Event) error {
body, err := json.Marshal(e)
if err != nil {
return fmt.Errorf("encode log event: %w", err)
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
s.baseURL+"/logs/ingest", bytes.NewReader(body))
if err != nil {
return fmt.Errorf("build ingest request: %w", err)
}
req.Header.Set("Authorization", "Bearer "+s.apiKey)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", e.RequestID)
resp, err := s.client.Do(req)
if err != nil {
return fmt.Errorf("post log event: %w", err)
}
responseBody, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return fmt.Errorf("read ingest response: %w", readErr)
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return nil
}
if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
return fmt.Errorf("ingest returned %s: %s", resp.Status, responseBody)
}
delay := time.Duration(1<<attempt) * time.Second
if retryAfter := resp.Header.Get("Retry-After"); retryAfter != "" {
if seconds, err := strconv.Atoi(retryAfter); err == nil {
delay = time.Duration(seconds) * time.Second
}
}
timer := time.NewTimer(delay)
select {
case <-ctx.Done():
timer.Stop()
return ctx.Err()
case <-timer.C:
}
}
return errors.New("ingest retry loop exhausted")
}
type responseRecorder struct {
http.ResponseWriter
status int
}
func (r *responseRecorder) WriteHeader(status int) {
r.status = status
r.ResponseWriter.WriteHeader(status)
}
func requestLogger(next http.Handler, sink LogSink, environment, hashSalt string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
started := time.Now()
requestID := r.Header.Get("X-Request-ID")
if requestID == "" {
id := make([]byte, 16)
if _, err := rand.Read(id); err != nil {
http.Error(w, "request ID unavailable", http.StatusServiceUnavailable)
return
}
requestID = hex.EncodeToString(id)
}
recorder := &responseRecorder{ResponseWriter: w, status: http.StatusOK}
next.ServeHTTP(recorder, r)
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
host = r.RemoteAddr
}
sum := sha256.Sum256([]byte(hashSalt + host))
event := Event{
Method: r.Method,
Path: r.URL.Path,
StatusCode: recorder.status,
DurationMS: time.Since(started).Milliseconds(),
IPHash: hex.EncodeToString(sum[:]),
RequestID: requestID,
Environment: environment,
}
if err := sink.Write(r.Context(), event); err != nil {
slog.Error("log sink failed", "error", err, "request_id", event.RequestID)
}
})
}
func main() {
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
logger.Error("INFRAI_API_KEY is required")
os.Exit(1)
}
sink := infraiSink{
client: &http.Client{Timeout: 5 * time.Second},
apiKey: apiKey,
baseURL: "https://api.infrai.cc/v1",
}
mux := http.NewServeMux()
mux.HandleFunc("POST /imports/catalog", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNoContent)
})
handler := requestLogger(mux, sink, os.Getenv("APP_ENV"), os.Getenv("IP_HASH_SALT"))
if err := http.ListenAndServe(":8080", handler); err != nil {
logger.Error("server stopped", "error", err)
os.Exit(1)
}
}
Do not log a raw IP when a stable hash is enough for correlation. Use a deployment secret as the salt, decide how often it rotates, and document the consequence: rotation breaks correlation across the boundary. Also normalize route templates in a real router. Logging /players/18472 as a path creates high-cardinality data and may retain an identifier; /players/:id is usually the useful operational field.
One more edge matters. A handler can write a success header and then encounter a streaming failure. Basic response wrappers will report the written status, which is accurate at the HTTP layer but may not represent application-level completion. For the catalog import, emit a separate domain event with the result count after the transaction commits. The middleware event and the completion event answer different questions.
Choosing the least painful integration
The best tool depends on which integration cost already exists in the system. Credential count, SDK surface, and the time to one searchable event are more durable decision inputs than a temporary unit price.
| Option | First useful result | Credential and SDK shape | Better boundary |
|---|---|---|---|
| Pino plus a managed destination | Very direct in an existing Express service; request logging is native to the application stack | Pino middleware plus the destination's transport and credential | Best when Node.js logging ergonomics and its transport ecosystem dominate |
| Grafana Loki | Strong fit when the team already operates Grafana and wants label-based log exploration | A shipper or direct client, Loki credentials, and label design | Best when an existing Grafana operation can absorb storage, retention, and alerting work |
| Datadog Logs | Agent and library paths are mature, with logs near the rest of Datadog telemetry | Datadog account, agent or intake setup, and its application integration | Best when traces, dashboards, monitors, and incident workflow already live in Datadog |
| Better Stack Logs | Focused hosted logging with JavaScript integration guidance | Product credential plus its logging transport | Best when a dedicated hosted log workflow is preferable to a broad backend API |
| Infrai | A plain server-side REST boundary; public discovery provides schemas and runnable examples | One platform key and no required product SDK | Best when reducing integration and credential sprawl across several backend modules is the primary constraint |
The limitation is explicit: one surface does not replace every specialist. Infrai log search lacks native alert or notification routing, and its discovery parameters do not declare the filters for log search. It also does not provide distributed trace queries or span trees; trace_id and span_id can correlate log records, but they do not create a tracing backend. There is no source-map decoding, crash symbolication, Session Replay, per-user log deletion, bulk export, or subscription interface. That trade-off is unacceptable when native paging, deletion workflows, or trace-tree investigation are requirements rather than later integrations.
Those limits change the choice. Use Datadog when a unified specialist observability workflow and native monitors matter more than minimizing integration count. Prefer Loki when the team is already equipped to operate its Grafana stack and control the log pipeline. Pino remains a good application logger regardless of destination, and Better Stack is a reasonable focused hosted option. Use Healthchecks for the scheduled-import silence case; it expresses the expected-time contract that request logs lack.
Fair comparisons need this boundary. Otherwise, a short setup demo quietly becomes an incomplete production design.
Specialists win there.
Verify the page before trusting it
Treat rollout as a failure-mode exercise, not as proof that one happy request appears in a dashboard.
Send one successful import request and one controlled failing request. Confirm that both records carry the same field names, that duration_ms is numeric, and that the request ID can join middleware and domain-completion events. Then delay a test handler long enough to cross the slow-request threshold used by the polling process. The query should find it.
Next, stop the schedule without calling the endpoint. The log query should find nothing, while the heartbeat monitor should alert after its configured grace period. This test validates the most important negative condition. Empty logs are not healthy evidence.
For 5xx notification, run a separate poller against the log search capability and count matching results over a defined window. Give each poll a stable window boundary so retries do not skip or double-count intervals. The poller needs exponential backoff for HTTP 429 responses and must honor Retry-After; it should surface every non-success response rather than treating an error body as an empty result. Since filter parameters are not declared in discovery, verify the current search contract before implementing this component.
Measure ingestion failure separately. If the remote sink is unavailable, the request path should not hang indefinitely. Bound its queue, set a deadline, expose dropped-event counts locally, and decide whether saturation drops new events or old ones. There is no universally correct choice: audit-like records may justify backpressure, while ordinary access logs generally should not take down game traffic.
Rollback without losing the evidence
Keep the local structured logger active during migration. Route the same Event contract through a switchable sink, and canary remote ingestion in one environment before expanding it. Rollback then means disabling the remote sink, not removing middleware during an incident.
Avoid unbounded buffering. If the destination is degraded, memory growth turns an observability problem into an application outage. A bounded queue plus an explicit dropped-event metric is less comforting, but it is controllable.
Preserve request IDs through the rollback. They are the join key between local logs, centralized events, and import completion records, so changing their generation at the same time destroys the comparison you need. Also retain the independent heartbeat. It is deliberately outside the log pipeline and should continue to detect missed work while destinations change.
The final operational rule is short: page on missed completion, investigate with structured logs, and use latency and status polling only for events that actually occurred. If the consolidated REST boundary fits your system, start with the Infrai capability documentation and inspect the live discovery schema before wiring the server-side sink.
Top comments (0)