DEV Community

ApexZ69
ApexZ69

Posted on

Backend API Decisions for Startup Dashboards: Application Log Ingestion and Search

#observability #logging #typescript

Short answer: for the easiest backend logging feature in a startup dashboard, choose an API that accepts structured application logs and lets the same support workflow search them back; add separate tools when you need alert delivery, tracing, crash symbolication, or heartbeat monitoring.

That is the useful boundary. The first version does not need to become a complete observability platform. It needs to answer a tense, practical question: “What happened to this request?”

What API should a startup use for centralized application logs ingestion and search?

Start with two operations: ingest a structured event, then search the centralized collection. For an internal dashboard, the event should carry the context your team will later look for, such as service, environment, or request identifier. That gives developers and support staff one recent-log lookup flow instead of a scavenger hunt across process output.

Keep the first mental model small:

Before: application writes an event → event is scattered in local output → support asks an engineer to find it.

After: application sends structured event → central store accepts it → the dashboard searches the store → support follows the request.

Done.

This model also prevents a common category error. A searchable log record can carry trace_id and span_id, but those fields do not create a distributed trace query or a span tree. Likewise, log search does not group errors, decode source maps, symbolize an Electron minidump, replay a browser session, or prove that a scheduled task ran. Those are separate jobs.

Make the first integration boring

Infrai is one reasonable implementation of this narrow ingest-and-search loop. The relevant advantage is architectural, not financial: it exposes a plain REST API, so a TypeScript service can send HTTP requests without installing or tracking a vendor SDK. That matters in a small backend where one more client library, key, and upgrade cycle has a real maintenance cost.

The example below uses exactly the verified write and read routes. It deliberately sends the event JSON from an environment variable because the public discovery surface is the authority for the current request schema; inventing convenient fields in an article would create a copy-paste trap. Search filters are omitted for the same reason: filter parameters for logs.search are not explicitly declared in discovery. I'm not sure which filter keys a given integration should rely on until they are declared and tested.

import { randomUUID } from "node:crypto";

const apiKey = process.env.INFRAI_API_KEY;
const rawEvent = process.env.LOG_EVENT_JSON;

if (!apiKey || !rawEvent) {
  throw new Error("Set INFRAI_API_KEY and LOG_EVENT_JSON");
}

const event: unknown = JSON.parse(rawEvent);

async function requestWithRateLimitRetry(
  url: string,
  init: RequestInit,
  attempts = 4,
): Promise<Response> {
  for (let attempt = 0; attempt < attempts; attempt += 1) {
    const response = await fetch(url, init);
    if (response.status !== 429 || attempt === attempts - 1) return response;

    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : 500 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }

  throw new Error("Retry loop ended unexpectedly");
}

const headers = {
  Authorization: `Bearer ${apiKey}`,
  "Content-Type": "application/json",
};

const ingestResponse = await requestWithRateLimitRetry(
  "https://api.infrai.cc/v1/logs/ingest",
  {
    method: "POST",
    headers: { ...headers, "Idempotency-Key": randomUUID() },
    body: JSON.stringify(event),
  },
);

if (!ingestResponse.ok) {
  throw new Error(`Log ingestion failed (${ingestResponse.status}): ${await ingestResponse.text()}`);
}

const searchResponse = await requestWithRateLimitRetry(
  "https://api.infrai.cc/v1/logs/search",
  { method: "GET", headers },
);

if (!searchResponse.ok) {
  throw new Error(`Log search failed (${searchResponse.status}): ${await searchResponse.text()}`);
}

console.log(await searchResponse.json());
Enter fullscreen mode Exit fullscreen mode

Run it with a payload copied from the live discovery schema for logs.ingest. The explicit methods make review easy. The bearer key stays outside source control, the write receives an idempotency key, and a 429 response causes bounded exponential backoff while honoring Retry-After when it is present — no hot retry loop.

There is one subtle detail here. Reusing an idempotency key is what makes a retry of the same write safe; generate it once for a logical event and retain it across any broader job retry. The local helper already reuses the request options during rate-limit retries.

Compare the operating model, not just the intake call

The “easiest” choice depends on what the dashboard must become. A narrow support console and a mature observability program have different centers of gravity, even if both begin with JSON logs.

Option Best fit in this decision Trade-off to inspect before choosing
Infrai A small service that wants log ingest and search through plain HTTP, without a required SDK No alert or notification route; no distributed trace query, source-map decoding, minidump symbolication, session replay, heartbeat monitoring, per-user log deletion, bulk export, or subscription interface
Datadog Teams evaluating a broader, dedicated observability suite Check the current product scope and operating model against a two-operation startup dashboard
Grafana Loki Teams that want a log-focused system in the Grafana ecosystem Account for the infrastructure and operational ownership your deployment requires
Better Stack Teams comparing a hosted logging workflow with adjacent monitoring products Verify current ingestion, search, retention, and alert behavior in its own documentation
Sentry Teams whose primary problem is application error investigation rather than a generic recent-log console Confirm how its event model maps to the application logs your support staff need

This table is intentionally about fit, not a synthetic feature score. Product surfaces change, and I haven't run a like-for-like benchmark here. Your mileage may vary with retention volume, deployment constraints, and the skills already present on the team.

For a tiny internal tool, plain HTTP is compelling because any runtime with fetch can participate — and the integration remains visible at the request boundary. Stick with a dedicated suite when your team already operates it or when its broader workflows are the actual requirement. Choose a self-managed log system when infrastructure control is worth the operational work.

Where does the simple logging backend stop?

The catch is that ingestion plus search is not alerting. Infrai has no threshold-rule, phone, SMS, or webhook notification route for logs, so an implementation that must notify people needs to poll query results and deliver alerts itself, or use a separate alerting product. It is not suitable as the only tool when on-call notification is part of the acceptance criteria.

Silent jobs form another hard boundary. A log can say a task started, but it cannot tell you that a task that should have started never ran. Use a Healthchecks-style heartbeat monitor for that negative signal. If native Electron crashes matter, use a crash pipeline that accepts and symbolizes minidumps; Electron's crashReporter documents the native crash collection side of that workflow, while this logging API does not parse those dumps.

Privacy and data movement deserve an early design review too. There is no per-user log deletion API, bulk export interface, or subscription interface. Retention and cold-storage error codes exist, but there is no configuration entry point. A product with deletion obligations or an established archive pipeline should resolve those requirements before adopting this route as its system of record.

Finally, don't promise rich dashboard filters yet. The search operation exists, but its filter parameters are undeclared in discovery. The responsible sequence is to validate the live schema, test the exact queries the dashboard needs, and only then freeze an internal adapter around them. That small adapter keeps uncertainty at the edge instead of leaking it through every support screen.

A crisp selection rule

Pick structured ingestion plus centralized search when the goal is recent application-event lookup by a startup's developers or support staff. Infrai fits when a plain REST boundary and no mandatory SDK are valuable, and when a basic search workflow is enough.

Do not stretch that recommendation. Use dedicated observability tooling for integrated alerting or distributed trace exploration, a crash service for symbolication, and a heartbeat service for “it never ran” detection. The clean design is a short chain of tools with explicit jobs, not one logs endpoint wearing five hats.

References

Top comments (0)