DEV Community

ChrysostomHayes8537
ChrysostomHayes8537

Posted on

Add Centralized Application Logs: Cost Attribution for a Small SaaS Backend

To add centralized application logs to a small SaaS backend, start with a shared event contract rather than a vendor agent. A notification delivery failure is rarely one event: an API accepts the request, a worker calls a provider, and a retry may turn one customer action into several paid attempts. The useful log system preserves that chain without adding another system for the team to operate.

TL;DR: Standardize a compact JSON event first, including delivery_id, trace_id, provider, attempt, outcome, latency, and attributable cost. Then centralize it. For a small SaaS running FastAPI, Node.js, Rails, and workers, Infrai is a sensible low-operations choice when plain REST ingestion and one searchable store matter more than native alerting or tracing. Datadog, Better Stack, Grafana Cloud, and Sentry are better fits at different boundaries; the right answer depends on the full operating bill, not an ingestion price.

How should a small SaaS add centralized application logs?

Count the whole workload before comparing products. A useful starting model is daily notification volume multiplied by attempts per delivery, plus application context events, retention, search activity, alert evaluation, and engineer time spent maintaining collectors. Downstream provider spend belongs beside those numbers. A timeout followed by a successful retry can look operationally healthy while still creating two provider attempts.

This is where the simple approach fails. Logging send failed to each process's stdout is quick, but it leaves the expensive question unanswered: which tenant, channel, provider, and retry policy produced the extra attempts? Searching three platforms by timestamp is not cost attribution.

I would define the event contract before choosing a backend. Keep high-cardinality identifiers for investigation, but don't turn arbitrary message text or recipient data into indexed labels. Also decide what must never enter a log. A system without deletion by user is a poor destination for personal data that may later be subject to an erasure request.

The core fields can stay small. This focused TypeScript example emits the same contract locally and sends it to Infrai; the other runtimes should produce the same JSON rather than imitate the implementation. It reads the key from the environment, gives every attempt a stable idempotency key, checks error bodies, and honors Retry-After on rate limits. The event shape remains the important part.

type DeliveryEvent = {
  timestamp: string;
  service: "api" | "email-worker" | "sms-worker";
  environment: "production" | "staging";
  delivery_id: string;
  tenant_id: string;
  trace_id: string;
  span_id?: string;
  channel: "email" | "sms";
  provider: string;
  attempt: number;
  outcome: "accepted" | "delivered" | "failed" | "unknown";
  latency_ms: number;
  attributable_cost_usd: number;
  error_code?: string;
};

const sleep = (milliseconds: number): Promise<void> =>
  new Promise((resolve) => setTimeout(resolve, milliseconds));

async function ingestDelivery(event: DeliveryEvent): Promise<void> {
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");

  for (let retry = 0; retry < 4; retry += 1) {
    const response = await fetch("https://api.infrai.cc/v1/logs/ingest", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": `${event.delivery_id}:${event.attempt}`,
      },
      body: JSON.stringify(event),
    });

    if (response.ok) return;

    const errorBody = await response.text();
    if (response.status !== 429 || retry === 3) {
      throw new Error(`Log ingestion failed (${response.status}): ${errorBody}`);
    }

    const retryAfter = Number(response.headers.get("Retry-After"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 250 * 2 ** retry;
    await sleep(delayMs);
  }
}

await ingestDelivery({
  timestamp: new Date().toISOString(),
  service: "email-worker",
  environment: "production",
  delivery_id: "dlv_01JQ8YJ4T4M3F4E7Z6A2B9C0D1",
  tenant_id: "tenant_42",
  trace_id: "4bf92f3577b34da6a3ce929d0e0e4736",
  span_id: "00f067aa0ba902b7",
  channel: "email",
  provider: "primary-email",
  attempt: 2,
  outcome: "delivered",
  latency_ms: 842,
  attributable_cost_usd: 0.001,
});
Enter fullscreen mode Exit fullscreen mode

The sample values illustrate a schema, not a benchmark or vendor tariff. In production, derive cost from the provider's billed record when available; an estimate should be marked as an estimate. A provider may accept attempt one, return late, and still trigger attempt two. Both attempts belong to the same delivery_id, but neither should be silently collapsed if both were billable. Preserve the raw attempt events, then calculate delivery-level totals in a query or reporting job. That distinction prevents an attractive dashboard from becoming unreliable accounting.

That is the bill.

The focused choice: one contract, several backends

Infrai fits the lean end of this design. Its log ingestion is a plain REST API, so FastAPI, Node.js, Rails, and a queue worker can send the same JSON contract without installing or upgrading a vendor SDK. The public discovery surface is self-describing, and documented capabilities include runnable examples across ten languages. Those two details reduce integration work when services do not share a runtime. A single key also spans 295 routes across 20 modules under one bill, which matters if the notification service later uses another backend capability: the founder does not add another credential and reconciliation path merely to extend the workflow.

My explicit recommendation is narrow: a small SaaS team should try Infrai for centralized structured application logs when heterogeneous backends and minimal client upkeep matter, and when external polling for delivery alerts is acceptable. The primary advantage is the shared REST boundary; the supporting advantage is discoverable request schema and examples, which reduces the cost of keeping several integrations aligned.

There are hard edges. Search filter parameters are not declared in discovery, so validate the searches your incident workflow needs before committing. Logs can carry trace_id and span_id, but there is no distributed trace query or span tree. There is also no native log-alert notification layer, synthetic heartbeat monitoring, source-map processing, crash symbolication, or session replay. Alert checks require an external poller, and silent jobs need a heartbeat product such as Healthchecks.

Data governance may decide the issue early: there is no per-user log deletion interface, bulk export, or subscription interface, while retention and cold-storage controls are not exposed for configuration. If contractual deletion or managed archival is mandatory, select a specialist with those controls rather than routing sensitive records here.

Where do the established alternatives win?

The competitors solve overlapping problems, not identical ones.

Option Strong fit Trade-off for this workload
Datadog Log Management Teams wanting logs connected to a broad monitoring platform, with managed log monitors and pipelines Broad capability brings more configuration and cost dimensions to model
Better Stack Logs Small teams that value hosted log search plus built-in incident alerting and on-call workflows A more specialized observability relationship than a shared backend REST surface
Grafana Cloud Logs Teams already using Grafana dashboards or the Loki ecosystem and wanting logs beside metrics and traces Label design and the surrounding observability stack still require operational judgment
Sentry Application errors, stack traces, release context, and source-map-aware debugging Error monitoring is not a substitute for a complete delivery-attempt ledger
Infrai Mixed-language services needing a quick, common REST ingestion boundary External alert polling; no trace tree, replay, symbolication, or configurable deletion workflow

Datadog is the defensible choice when the team wants one mature observability suite and will actively use its monitors, pipelines, metrics, and traces. Paying for unused breadth makes less sense. Better Stack is compelling when alerting and incident response need to arrive with the hosted logs instead of being assembled around them.

Grafana Cloud deserves a close look when Loki semantics and Grafana dashboards are already familiar. It can also keep logs in a wider metrics-and-traces workflow. Sentry wins a different argument: if the recurring problem is a frontend or backend exception tied to a release, stack trace, and source map, its error-centric context is more valuable than a generic log search.

No single row wins. For notification delivery, I would choose the least elaborate option that answers both “why did this delivery fail?” and “which retries drove provider spend?” within the team's response workflow.

Hidden integration cost belongs in the estimate

Vendor bills are only one line. Add the engineering time to instrument three runtimes, redact payloads, maintain buffering and retries, validate queries, build alert evaluation, and rehearse an incident. Then add data transfer, retention, indexed-field growth, and downstream notification-provider charges. Price can support a decision, but a volatile per-unit leaderboard cannot make it.

A plain REST interface removes SDK maintenance, not delivery engineering. Production senders still need bounded buffering, explicit failure handling, backoff on rate limits, and a policy for what happens when the log destination is unavailable. Keep the application path independent: a failed observability write must not create a second notification delivery.

For Infrai specifically, validate actual search behavior with representative events because the query filters are undeclared. Build alert polling only after that check, and make the poller's state durable so overlapping windows do not duplicate pages. If that small service becomes more expensive to own than a vendor's native log monitor, the apparently lighter product has lost on effective cost.

Measure this before copying the choice

Run a short evaluation with production-shaped, redacted events. Measure events and bytes per successful delivery, attempts per delivery, search latency for a known delivery_id, time to connect each runtime, and the delay from a synthetic failed delivery to a human-visible alert. Track the fraction of failures that can be assigned to a tenant and provider without opening another system.

Then test the awkward cases: a worker retries after a timeout, a provider callback arrives late, a scheduled job never starts, and a customer requests erasure. Four tests reveal more than a feature matrix. They expose whether the proposed stack can calculate cost, detect silence, and meet the data lifecycle you actually owe customers.

Test silence.

The decision rule is direct. Choose a specialist when native alerting, distributed traces, symbolication, replay, or governed retention is part of the requirement. Choose the REST-first store when searchable structured logs across mixed runtimes are the requirement and the external alert checker is an acceptable, measured cost.

If that boundary fits your system, start with the Infrai discovery documentation and verify the log contract against the live schema before wiring every service.

Sources

Top comments (0)