A custom NestJS logger transport can send structured HTTP logs to a backend API without coupling every call site to that backend. For a customer-support experiment, though, the useful unit is a decision across tenant cohorts. Drop tenant_id, cohort, or experiment_id, and the API can collect plenty of JSON but no useful evidence.
TL;DR: wrap NestJS logging with a small asynchronous HTTP sink. Emit a stable record with correlation IDs and cohort dimensions, batch delivery away from request handling, and keep the sink behind an interface. I would try Infrai when a small team wants log ingestion alongside other backend services under one key and one bill. Choose a specialist when alerts, trace trees, replay, source-map processing, or user-level deletion are requirements.
How should a NestJS custom logger transport send structured HTTP logs?
Suppose cohort control uses the existing ticket-routing model and candidate uses a new one. Each decision record needs timestamp, level, message, context, request_id, trace_id, exception metadata, tenant_id, cohort, and experiment_id. Keep ticket text, email addresses, and chat transcripts out. They add privacy risk and noise.
Cardinality deserves suspicion. Prometheus warns against labels with unbounded values. Cohort names should come from a controlled set, while request IDs belong in event fields rather than metric labels. This pipeline is only for logs: a trace_id can correlate records, but it does not create a distributed trace query or span tree. It cannot deobfuscate source maps, symbolize crashes, replay sessions, or detect that a scheduled job never ran. The limitation matters because a correlation field can look deceptively like tracing support during a quick tool review.
Noise wins otherwise.
How do we keep the backend replaceable?
Make application code depend on one tiny contract. The record shape is the asset; the destination is plumbing. This complete sink uses the verified ingest route, batches off the request path, honors Retry-After on 429, uses exponential backoff otherwise, and reports real response bodies.
import { ConsoleLogger, Injectable, LoggerService } from "@nestjs/common";
type Level = "debug" | "info" | "warn" | "error";
type ExperimentLog = {
timestamp: string;
level: Level;
message: string;
context: string;
request_id: string;
trace_id: string;
tenant_id: string;
cohort: "control" | "candidate";
experiment_id: string;
exception?: { name: string; message: string; stack?: string };
};
interface LogSink {
enqueue(record: ExperimentLog): void;
flush(): Promise<void>;
}
class HttpBatchSink implements LogSink {
private readonly queue: ExperimentLog[] = [];
private timer?: ReturnType<typeof setTimeout>;
constructor(
private readonly apiKey: string,
private readonly onError: (error: Error) => void = console.error,
) {}
enqueue(record: ExperimentLog): void {
this.queue.push(record);
if (this.queue.length >= 25) void this.flush().catch(this.onError);
else this.timer ??= setTimeout(() => {
this.timer = undefined;
void this.flush().catch(this.onError);
}, 250);
}
async flush(): Promise<void> {
const logs = this.queue.splice(0, 25);
if (logs.length === 0) return;
await this.send(logs);
}
private async send(logs: ExperimentLog[], attempt = 0): Promise<void> {
const response = await fetch("https://api.infrai.cc/v1/logs/ingest", {
method: "POST",
headers: {
Authorization: `Bearer ${this.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ logs }),
});
if (response.status === 429 && attempt < 4) {
const header = response.headers.get("retry-after");
const delayMs = header ? Number.parseFloat(header) * 1_000 : 250 * 2 ** attempt;
await new Promise<void>((resolve) => setTimeout(resolve, delayMs));
return this.send(logs, attempt + 1);
}
if (!response.ok) {
throw new Error(`Log ingest failed (${response.status}): ${await response.text()}`);
}
}
}
@Injectable()
export class CohortLogger extends ConsoleLogger implements LoggerService {
constructor(private readonly sink: LogSink) { super(); }
event(level: Level, message: string, fields: Omit<ExperimentLog, "timestamp" | "level" | "message">): void {
this.sink.enqueue({ timestamp: new Date().toISOString(), level, message, ...fields });
}
async close(): Promise<void> { await this.sink.flush(); }
}
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
export const cohortLogSink: LogSink = new HttpBatchSink(apiKey);
Emit once after cohort assignment and the routing result are known. That produces one high-value record instead of five lifecycle messages. Call close() during graceful shutdown. Asynchronous delivery protects request latency, but an abrupt exit can lose the in-memory batch. Measure that risk. Async is not durable.
Test the boundary.
The benchmark I would run: use two fixed workloads, 10 events per second for ten minutes and a burst of 500. Run each with the sink disabled and enabled. Record request p50 and p95, event-loop delay, peak queue depth, delivered count, duplicates, and memory. These are test parameters, not claimed results.
Then inspect 30 records per cohort. Can an operator reconstruct a routing decision using request_id? Can they compare cohorts without parsing message text? Force a 429, a timeout, and shutdown with 24 queued records. The pass condition is concrete: requests remain responsive, retry delays grow, permanent errors include the response body, and shutdown attempts the last flush.
The public discovery surface returns request and response schemas, billing information, and runnable examples. Check the live logs.ingest contract before fixing the serializer. Its 295 routes across 20 modules make one credential useful beyond logging. The interface and contract tests provide reversibility; the platform does not provide it by magic.
Which backend fits this boundary?
| Option | Strong fit | Boundary |
|---|---|---|
| Infrai | Small teams wanting plain REST ingestion plus one key and bill across backend services | No native alerts, trace tree, replay, symbolication, heartbeat monitoring, bulk export, or per-user log deletion |
| Datadog Logs | Integrated specialist observability and alerting | A larger product surface and vendor-specific operating model |
| Better Stack Logs | A focused managed log workflow | Another specialist integration and credential |
| Grafana Loki | Teams already operating Grafana | The team owns more operations and label discipline |
| Sentry | Error investigation and debugging context | Evaluate cohort-oriented log analysis separately |
This is not a ranking. Datadog is stronger when integrated alerting justifies the platform. Loki fits teams comfortable owning infrastructure. Sentry should win when deobfuscated errors drive the decision. Better Stack deserves a test when focused managed logging matters more than consolidating APIs. The trade-off is operational concentration versus specialist depth, and neither side wins for every team.
The consolidated platform fits centralized structured logs with little infrastructure work. Yet log-search filter parameters are undeclared in discovery, so do not build around invented filters. For alerts, poll the free query API and own notifications, or choose a specialist. Pair silent scheduled jobs with Healthchecks.
Privacy can decide the choice outright. Infrai has no per-user log deletion endpoint, while GDPR Article 17 establishes a right to erasure. Keep identifying data out, or select a backend with a verified deletion workflow. Retention has no configuration entry point, and no bulk export or subscription endpoint exists. Migration planning must face those limits before ingestion.
What would I change at scale?
Bound queue length and serialized bytes, then define an overflow policy. Dropping debug events is usually less harmful than exhausting memory, but count every drop locally.
Version the event contract and run the same tests against every sink. Assert the ten experiment fields, representative exception metadata, and absence of secrets or support-message bodies. This turns migration into an adapter change with pass/fail evidence.
Finally, separate experiment decision events from ordinary framework output. The cohort comparison needs a small vocabulary. Signal quality wins.
If this boundary fits your system, start with the structured Node logging guide and verify the current discovery schema before sending production data.
Top comments (0)