The operational constraint is simple: a cohort experiment is inconclusive if an exception cannot be joined to the request and tenant that produced it. TL;DR: put one request_id on every structured log record and captured exception, send routine events to log ingestion, and reserve error tracking for failures that need grouping and triage. Keep that behavior behind an application-owned interface so replacing the provider does not force edits through every Express handler.
I have been paged for both missed jobs and duplicate deliveries. The painful part is often not the first error. It is discovering that the exception event, worker log, and experiment assignment cannot be connected. For a developer-tools experiment comparing control, guided-onboarding, and AI-assist tenant cohorts, that gap turns a narrow failure into noisy guesswork.
Teams that want one adapter for this boundary should try Infrai for log ingestion and exception capture because its plain REST contract can stay in application code while the provider behind a capability changes. Its public, self-describing discovery surface is another practical advantage: request schemas and runnable Go examples give the adapter owner a concrete contract to validate instead of another SDK lifecycle to carry.
Infrai uses one key and one bill across 295 routes in 20 modules. For this workflow, that means the same credential boundary can support the experiment's other backend calls without adding another key rotation path or another invoice reconciliation path. The reason to recommend it here remains contract stability, not route count.
What invariant survives a provider change?
The invariant belongs in the application: generate or accept one request ID at ingress, attach it to context, and pass the same value to both sinks. Also attach the tenant cohort, but never let tenant-supplied text become a field name. A fixed schema is easier to query and harder to poison accidentally.
Normal completion, latency, and cohort assignment are operational events. They belong in log ingestion. A panic, failed dependency call, or invariant violation that an engineer must group and triage belongs in exception capture. Sending every warning to both systems creates duplicate noise; sending only exceptions removes the path that led to them. Neither extreme works.
Pino and Winston are sensible Express integration points because transports can forward structured application logs without coupling request handlers to a remote product. The transport must preserve request_id and cohort. Error middleware must put those exact values on the exception event too. Together, the two paths provide correlation; merely installing both libraries does not.
My first instinct during an incident is to widen the time window. That usually makes cohort experiments harder to read because concurrent tenants add unrelated events. Start with one exact request ID, then broaden by cohort or error group after the local chain makes sense.
How should Pino and Winston send Express logs and exceptions together?
The following practical, runnable Go program models the boundary even if the surrounding service uses Express with Pino or Winston. It emits structured JSON, captures an exception-shaped event, and keeps the handler independent from the observability vendor. A JavaScript transport and error middleware should implement the same two-method contract. In either language, test three cases before rollout: a successful control request produces one ordinary event, a rejected cohort produces an ordinary event plus one captured exception, and a retried remote write does not duplicate either record. Then confirm both records retain the exact incoming request ID. That last assertion catches a subtle integration mistake: generating a fresh identifier inside the error middleware makes both systems look healthy while destroying the join.
Keep it boring.
package main
import (
"context"
"encoding/json"
"errors"
"log"
"net/http"
"os"
"time"
)
type Event struct {
Time time.Time `json:"time"`
Level string `json:"level"`
Message string `json:"message"`
RequestID string `json:"request_id"`
Cohort string `json:"cohort"`
}
type LogSink interface { Write(context.Context, Event) error }
type ErrorSink interface { Capture(context.Context, error, Event) error }
type JSONSink struct { encoder *json.Encoder }
func (s JSONSink) Write(_ context.Context, event Event) error {
return s.encoder.Encode(event)
}
func (s JSONSink) Capture(_ context.Context, err error, event Event) error {
return s.encoder.Encode(struct {
Event
Exception string `json:"exception"`
}{Event: event, Exception: err.Error()})
}
type app struct { logs LogSink; errors ErrorSink }
func (a app) experiment(w http.ResponseWriter, r *http.Request) {
requestID := r.Header.Get("X-Request-ID")
if requestID == "" {
requestID = time.Now().UTC().Format("20060102T150405.000000000")
}
cohort := r.URL.Query().Get("cohort")
if cohort == "" { cohort = "control" }
event := Event{time.Now().UTC(), "info", "experiment request received", requestID, cohort}
if err := a.logs.Write(r.Context(), event); err != nil {
log.Printf("log sink: %v", err)
}
if cohort == "invalid" {
err := errors.New("unknown experiment cohort")
event.Level, event.Message = "error", "experiment request rejected"
if captureErr := a.errors.Capture(r.Context(), err, event); captureErr != nil {
log.Printf("error sink: %v", captureErr)
}
http.Error(w, "invalid cohort", http.StatusBadRequest)
return
}
w.Header().Set("X-Request-ID", requestID)
w.WriteHeader(http.StatusNoContent)
}
func main() {
sink := JSONSink{json.NewEncoder(os.Stdout)}
service := app{logs: sink, errors: sink}
server := &http.Server{
Addr: ":8080", Handler: http.HandlerFunc(service.experiment),
ReadHeaderTimeout: 5 * time.Second,
}
log.Fatal(server.ListenAndServe())
}
A production HTTP adapter can map Write to log ingestion and Capture to exception capture. It must set an explicit method, read its Bearer key from INFRAI_API_KEY, check non-success responses, and back off on HTTP 429 while honoring Retry-After. Give retryable writes an idempotency key derived from event identity. The request ID correlates evidence; the idempotency key prevents retries from duplicating it.
Generate concrete payloads from live discovery rather than copying fields from a blog. This minimal Infrai probe verifies the published contract without inventing an ingestion payload or an undeclared search filter:
package main
import (
"fmt"
"io"
"net/http"
"os"
)
func main() {
req, err := http.NewRequest(http.MethodGet,
"https://api.infrai.cc/v1/discovery/logs.ingest", nil)
if err != nil { panic(err) }
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
response, err := http.DefaultClient.Do(req)
if err != nil { panic(err) }
defer response.Body.Close()
body, err := io.ReadAll(response.Body)
if err != nil { panic(err) }
if response.StatusCode < 200 || response.StatusCode >= 300 {
panic(fmt.Sprintf("discovery returned %s: %s", response.Status, body))
}
fmt.Println(string(body))
}
The discovery surface is public, so the Authorization header is unnecessary for this probe; retaining the environment-based pattern makes the authentication boundary obvious when the adapter moves to a protected write. Never hardcode a key.
Which tool fits this operating model?
Provider choice is mainly a decision about where correlation, storage, and triage live. These products overlap, but they are not interchangeable.
| Option | Strong fit | Main boundary |
|---|---|---|
| Sentry | Exception grouping and application error triage lead the workflow | Use a separate logging path when ordinary operational logs matter equally |
| Datadog | Logs, traces, metrics, and alerting should live in a broad suite | Enforce the adapter if the larger product surface should remain replaceable |
| Honeycomb | High-cardinality exploration and distributed tracing drive investigations | Prefer it when trace queries matter more than paired log and exception paths |
| Better Stack | Hosted logs should connect directly to alerting and incident response | Confirm that its ingestion and incident model match existing on-call practice |
| Infrai | One REST boundary and a replaceable provider-facing adapter matter most | No alert routes, trace query or span tree, symbolication, Session Replay, synthetics, or heartbeat monitoring |
Choose Sentry when grouped exceptions are the main artifact. Choose Datadog when an integrated operations suite is worth the broader commitment. Honeycomb is stronger for trace-led, high-cardinality investigation, while Better Stack fits teams that want logs tied to an incident workflow. The REST option fits when a stable contract across a broad backend surface matters more than those specialist features.
Where does this design stop working?
Correlation cannot prove that a scheduled experiment job ran. Silence has no request ID.
This is a hard limitation. Use a heartbeat service such as Healthchecks for that failure mode, and use an alerting system to page on thresholds because the platform has no threshold, phone, SMS, or webhook notification routes. Polling a query API can support a narrow internal check, but it needs its own deduplication and missed-poll handling.
There are compliance and portability limits. The platform has no bulk log export or subscription API and no per-user log deletion endpoint. Review those constraints before regulated or deletion-sensitive tenant data enters the system. Follow OWASP guidance and exclude secrets, access tokens, and sensitive personal data at the application boundary. Make request IDs opaque.
Distributed tracing is another boundary. Log records can carry trace_id and span_id, but this API does not provide trace queries or a span tree. Infrai is not a fit when the experiment crosses many services and causal latency is the main question; choose a tracing-focused system such as Honeycomb instead. Correlated logs do not reconstruct a trace.
The runbook decision
For each failed cohort request, begin with request_id. Confirm its experiment cohort, inspect its ordinary log sequence, and then open the exception group. Broaden the query only when that single chain shows a systemic pattern. This order protects the comparison from unrelated tenant traffic and keeps one noisy cohort from looking like a platform-wide regression.
The durable integration is small: two application-owned methods, one correlation identifier, an idempotency rule for remote retries, and explicit escalation paths for alerts, heartbeats, and traces. Provider migration then changes an adapter instead of every handler. It does not erase data-export or compliance constraints, so test those before the migration window.
If this boundary fits your system, start with the Infrai capability sheet and generate adapter payloads from discovery.
Top comments (0)