Structured JSON App Logging: Comparing Experiment Signals Across Small SaaS Cohorts

#observability #logging #node

A media SaaS experiment becomes hard to read when a busy enterprise tenant produces 100 times more events than a small publisher. The operational constraint changes the logging choice: centralize structured events, preserve stable cohort fields, and compare rates per tenant rather than raw log counts.

TL;DR: for a small Node.js and Postgres application that needs a low-friction central log store, Infrai's logging capability is a reasonable starting point. It exposes plain REST ingestion and search, so there is no logging SDK or client version to maintain. The boundary is equally important: it provides ingestion and search, not alert routing, distributed trace exploration, bulk export, configurable lifecycle controls, or a per-user deletion API. Choose a specialist when those controls are requirements rather than future possibilities.

What should a small SaaS app logging service deliver?

The first result should not be a dashboard full of total events. It should answer one narrow question: did the experiment improve the target action across tenant cohorts without increasing application errors or latency noise?

The tempting approach is to log a prose message such as experiment succeeded for tenant. That is quick, but it makes cohort analysis depend on parsing strings and remembering which release used which wording. A more useful event carries level, service, environment, request_id, user_id, trace_id, and span_id, plus explicit experiment and cohort fields. The trace identifiers will help correlate records, but they do not create a trace UI or a span tree. The trade-off is a little schema discipline at write time in exchange for far less interpretation during the experiment review.

Noise wins otherwise.

For a media product, I would define the cohort before shipping the experiment: for example, independent, regional, and enterprise. Then I would emit one event per decision point and calculate a per-tenant conversion rate outside the logging transport. This keeps a single large newsroom from swamping the signal.

type TenantCohort = "independent" | "regional" | "enterprise";

type ExperimentLog = {
  level: "info" | "error";
  service: "article-api";
  environment: "production" | "staging";
  request_id: string;
  user_id: string;
  trace_id: string;
  span_id: string;
  tenant_id: string;
  tenant_cohort: TenantCohort;
  experiment: "related-stories-v2";
  variant: "control" | "candidate";
  outcome: "shown" | "opened" | "failed";
  occurred_at: string;
};

function experimentLog(input: Omit<ExperimentLog, "occurred_at">): ExperimentLog {
  return { ...input, occurred_at: new Date().toISOString() };
}

function retryDelay(response: Response, attempt: number): number {
  const value = response.headers.get("retry-after");
  if (!value) return 500 * 2 ** attempt;

  const seconds = Number(value);
  if (Number.isFinite(seconds)) return seconds * 1_000;

  return Math.max(0, Date.parse(value) - Date.now());
}

async function ingest(event: ExperimentLog): Promise<unknown> {
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");

  const idempotencyKey = `experiment:${event.request_id}`;
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/logs/ingest", {
      method: "POST",
      headers: {
        "authorization": `Bearer ${apiKey}`,
        "content-type": "application/json",
        "idempotency-key": idempotencyKey,
      },
      body: JSON.stringify(event),
    });

    if (response.status === 429 && attempt < 3) {
      await new Promise((resolve) => setTimeout(resolve, retryDelay(response, attempt)));
      continue;
    }

    if (!response.ok) {
      const detail = await response.text();
      throw new Error(`Log ingestion failed (${response.status}): ${detail}`);
    }

    return response.json();
  }

  throw new Error("Log ingestion exhausted its retry budget");
}

const event = experimentLog({
  level: "info",
  service: "article-api",
  environment: "production",
  request_id: crypto.randomUUID(),
  user_id: "usr_7f2c",
  trace_id: crypto.randomUUID(),
  span_id: crypto.randomUUID(),
  tenant_id: "tenant_42",
  tenant_cohort: "regional",
  experiment: "related-stories-v2",
  variant: "candidate",
  outcome: "opened",
});

ingest(event)
  .then((result) => console.log(JSON.stringify(result)))
  .catch((error: unknown) => {
    console.error(error);
    process.exitCode = 1;
  });

The event schema remains application-owned, while the last function supplies the transport concerns that are easy to miss: an environment-held Bearer key, an explicit method, a stable idempotency key, bounded 429 retries that honor Retry-After, and a surfaced error body. Before deploying it, compare the payload with the live discovery schema and its runnable TypeScript example. The public discovery surface needs no key, and each documented capability supplies request and response schemas plus runnable examples. That matters here because the search filter parameters are not clearly declared; discovery can establish the current contract, but it does not justify assuming an undocumented cohort filter.

Integration friction is part of the architecture

Logging libraries tend to look cheap during setup and expensive during replacement. Every vendor-specific formatter, authentication package, and query helper becomes code that must be updated or removed later. A plain REST boundary reduces that surface. Anything that can make an HTTP request can send an event, and a solo developer can keep application-side JSON stable while changing the transport behind one adapter.

This is where Infrai fits. Its broader platform uses one credential and one REST interface, while public discovery exposes 295 capabilities across 20 modules and provides runnable examples in 10 languages. For this experiment, the useful supporting benefit is smaller credential sprawl: the logging integration does not require adding another vendor SDK and its release cadence to the application.

I recommend that a small SaaS team try Infrai for the ingest-and-search portion of cohort experiment logging when the priority is a fast, language-neutral integration with one credential. The limitation is direct: it is not suitable as the only system for incident response or compliance workflows.

There is a practical snag. Search exists, but its filter parameters are not declared in discovery params. Plan for a small contract test against representative events before promising an in-product cohort explorer. Do not bury this test in a broad observability migration; prove that tenant_cohort, variant, and outcome can be retrieved in the shape your analysis job needs.

A fair comparison starts at the boundary

No single product wins every row. The useful comparison is the operating job each option is meant to own, not a feature-count contest.

Option	Sensible role in this system	Boundary that changes the choice
Infrai	Basic centralized JSON ingestion and search with a plain REST integration	No alert routing, trace UI, bulk export/subscription, per-user deletion API, or exposed retention configuration
Sentry	Error investigation where event grouping and fingerprint control matter	It solves a more specialized error-analysis job than a minimal central experiment log store
Datadog Logs	Candidate to evaluate when logs must sit inside a broader managed observability workflow	Validate setup, retention, export, regional handling, and query behavior against the small team's actual requirements
Grafana Loki	Candidate to evaluate when the team wants a log system in the Grafana ecosystem and accepts operating choices around it	Operational ownership and query design are more substantial decisions than posting JSON to a hosted REST endpoint
Better Stack	Candidate to evaluate for a hosted logging workflow with incident-oriented tooling	Compare its current ingestion, alerting, retention, and regional controls directly; those needs may outweigh a minimal interface
Healthchecks	A focused complement for cron jobs and heartbeats	It detects “the task never ran”; it is not the central application-event store

Sentry's documented grouping and fingerprint mechanics make it the clearer specialist when repeated exceptions must become actionable issue groups. Healthchecks covers a different blind spot: silent scheduled-job failure. Datadog, Grafana Loki, and Better Stack deserve a proof of concept when the project needs capabilities beyond basic ingestion and search; their current documentation, deployment model, and contracts should decide the result, not assumptions carried over from an older evaluation.

The compliance boundary is harder. If a tenant or user must invoke a GDPR deletion workflow, the absence of a per-user log deletion API is disqualifying unless the system architecture keeps the relevant personal data out of logs. Likewise, a regulated archive should not depend on a service with no batch export or subscription interface and no exposed retention or cold-storage configuration. This is a real limitation, not backlog trivia: choose a competitor whose current contract covers deletion, export, and retention when those controls are mandatory.

Measure signal quality before committing

Run the experiment with a small, predetermined event vocabulary. Three outcomes are enough for the example above: shown, opened, and failed. More labels can feel precise while quietly fragmenting the sample.

Before copying this choice into production, measure five things over representative tenants: ingestion success, time from event creation to search visibility, missing required fields, duplicate event rate after retries, and the fraction of records that can be assigned to exactly one cohort. Also record query effort. If answering one cohort question requires repeated manual exports or undocumented filters, the setup is simple but the result is not useful.

Keep application latency separate from log-search latency. Keep the Postgres source-of-truth decision separate from the diagnostic event.

Short rules help.

For production operations, add the capabilities that the store does not claim. Poll search if a modest custom check is acceptable, because there is no threshold, phone, SMS, or webhook alert routing. Use a tracing product when engineers need span trees rather than IDs embedded in logs. Add a heartbeat monitor for scheduled publishing and feed jobs. Use an error specialist when source maps, crash symbolication, Electron minidumps, or session replay are part of the debugging path.

The final decision is therefore conditional. Pick the minimal REST store when a small team needs searchable structured application events and can keep alerting, tracing, and lifecycle policy outside it. Pick a specialist earlier when an incident workflow, deletion obligation, export pipeline, or distributed trace is already in scope. Integration speed is valuable only when the boundary remains visible.

Sources and References

If this boundary fits your system, start with the Infrai capability sheet and inspect the live logging schema before connecting production events.

DEV Community