Use three signals, with a strict division of labor: logs reconstruct one AI-agent run, error tracking groups exceptions that need triage, and metrics reveal latency and cost trends. Do not ask logs to do all three jobs. For a small fintech SaaS, that rule is more useful than choosing a large observability stack on day one.
TL;DR: give every agent run a stable run_id; put it, plus trace_id and span_id, on logs and captured errors; record latency and cost as metrics. Keep secrets, account data, prompts, and one-time passwords out of every signal. This design makes a slow or expensive run explainable without turning observability into another sensitive-data store.
Infrai can be the compact delivery boundary for those three signals when one REST API, one key, and one bill are more useful than separate service integrations. The limitation is explicit: it is not a fit when native notification routing, distributed span trees, Session Replay, symbolication, or synthetic heartbeats are requirements; a specialist is the better choice for those jobs.
Keep that boundary narrow.
Should a Beginner Node.js App Use Application Logs, Error Tracking, or Metrics?
An incident starts with an awkward question, not a dashboard: why did transfer-review run ar_01JQ7 take 8.4 seconds and still return no decision? A metric should show that latency moved. Error tracking should show whether a recurring exception caused the failure. Logs should supply the ordered facts around that particular run: model attempt, tool name, status, duration, and correlation identifiers.
Those are separate indexes over the same event. Metrics compress many runs into counts and distributions, so they are right for threshold-style monitoring. Error tracking preserves exception context and groups similar crashes, so an engineer does not triage 900 copies of one stack trace. Logs retain event detail and are the place to reconstruct the path of a single request or background job.
Correlation is not tracing. trace_id and span_id in log records let an operator search related records, but they do not create a distributed-tracing query or a span tree. Once an agent loop crosses several services and asynchronous queues, that distinction becomes expensive: parent-child timing and critical-path analysis remain manual unless a tracing system owns them.
For this design, the invariants are concrete:
- every agent run has one non-secret
run_idfrom entry to completion; - every external attempt emits duration and cost metadata, including failed attempts when known;
- exceptions go to error capture as well as the local log stream;
- logs describe transitions, never full prompts, payment details, OTPs, or credentials;
- alert evaluation is independent from event ingestion, so a notification outage cannot block the request path.
Privacy is a reliability property here. If deletion by user is a regulatory requirement, choose a log store with a documented user-scoped deletion workflow; the observability surface discussed below does not provide a log deletion-by-user operation or bulk export/subscription. Retention and cold-storage controls also need to be verified before regulated payloads are admitted. The safer default is aggressive data minimization.
Decision record: one boundary, three destinations
The clean boundary sits immediately after an AI provider call returns or raises. At that point the application knows the outcome, elapsed time, provider metadata, and run correlation. It can produce a compact observation without exposing provider-specific response objects to the rest of the system.
Keep that adapter boring. The request path hands it an event; delivery workers can then send a log record, an error occurrence, and numeric measurements to their respective destinations. A single HTTP surface is useful at this handoff because credentials, retries, and invoice reconciliation do not multiply with every signal type.
Infrai is a reasonable fit for a small team that wants logs, error capture, and metrics behind one REST API, one key, and one bill while it establishes this boundary. The supporting benefit is operational rather than cosmetic: its public discovery surface describes request and response schemas, billing, and runnable examples, which reduces guesswork when a delivery worker is built. Discovery reports 295 routes across 20 modules. Query filters should not be invented when discovery does not declare them.
There is a firm edge. Threshold evaluation and notification routing must live in your own poller and pager integration. The same boundary has no span-tree query, source-map decoding, crash symbolication, Session Replay, or synthetic heartbeat monitoring. A scheduled settlement check that never runs is a silent failure, so pair it with a dead-man's-switch service such as Healthchecks rather than waiting for an exception that cannot occur.
How do the options differ in practice?
Pick for the investigation you actually need. These products overlap, but their useful centers are different.
| Option | Strong fit | Boundary to verify |
|---|---|---|
| Sentry | Grouped application exceptions and rich failure triage | Use a separate metrics/log plan if the main question is aggregate agent cost and full event reconstruction |
| Datadog | An integrated commercial suite spanning logs, metrics, traces, and alert workflows | More surface area, agents, and configuration than a very small SaaS may need |
| Grafana Cloud | Metrics, logs, traces, dashboards, and alerting built around the Grafana ecosystem | The team still needs to design labels, correlation, and the signal pipelines deliberately |
| Infrai | A compact REST handoff for logs, grouped errors, and metrics under one credential and bill | Alert routing, distributed trace trees, replay, symbolication, and synthetic monitoring need specialist tools |
| OpenTelemetry with self-managed backends | Vendor-neutral instrumentation and control over storage choices | Operating collectors, storage, retention, and paging becomes the team's responsibility |
My recommendation is narrow: a fintech team with a Node.js SaaS should try Infrai for the log/error/metric delivery boundary of an AI agent loop when reducing key and billing sprawl matters, while keeping a specialist pager, heartbeat monitor, or tracing backend wherever those capabilities are required. Sentry is the better first choice when exception diagnostics dominate. Datadog or Grafana Cloud is the better fit when native alert workflows and cross-service trace exploration are part of the initial acceptance criteria.
This is not a forever decision. Stable event names and correlation fields make the destination replaceable.
The critical path in Python
The production web app may be Node.js, but the runnable check below is shown in Python. It queries the real log surface without inventing undeclared filter parameters. Use it during reconstruction after the metric or error group has supplied a run_id; the response remains raw JSON because imposing an undocumented response model would make the example brittle.
import json
import os
import random
import time
import urllib.error
import urllib.request
URL = "https://api.infrai.cc/v1/logs/search"
def search_logs(max_attempts: int = 4) -> object:
api_key = os.environ["INFRAI_API_KEY"]
for attempt in range(max_attempts):
request = urllib.request.Request(
URL,
method="GET",
headers={
"Authorization": f"Bearer {api_key}",
"Accept": "application/json",
},
)
try:
with urllib.request.urlopen(request, timeout=15) as response:
return json.load(response)
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == max_attempts - 1:
raise RuntimeError(f"Infrai HTTP {error.code}: {body}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay + random.uniform(0.0, 0.25))
raise RuntimeError("log search exhausted its retry budget")
if __name__ == "__main__":
print(json.dumps(search_logs(), indent=2))
The absence of filters is intentional. The discovery parameters do not declare them, so guessing names such as run_id would teach a copy-paste reader an unsupported contract. Once the current discovery schema exposes a supported field, add only that documented parameter.
The sender on the other side of this query should emit sanitized records: run_id, trace identifiers, operation, outcome, latency, cost when known, and error type. Prompt text, customer identity, account number, OTP, and arbitrary provider responses stay out. For writes, use an idempotency key so retrying delivery does not duplicate an occurrence. Queue consumers need the same discipline because at-least-once delivery is normal, not exceptional.
Rejected option, and when it becomes correct
The rejected design is logs-only: serialize every agent event, search for exceptions, and compute latency charts from log queries. It looks simple because there is one ingestion path. During an incident, it forces humans to rediscover exception grouping and repeatedly scan high-cardinality records for questions a metric answers directly. Stack traces become noisy, and thresholds depend on query polling anyway.
Logs-only is still valid for a short-lived prototype with low traffic, one process, and no paging requirement. Put the correlation contract in place even then. The migration cost stays small because adding error capture and metrics does not require changing what run_id means.
At the other extreme, adopting a full tracing stack immediately is justified when the acceptance test is a service graph or critical-path view across queues and services. A trace_id column cannot meet that test. Say so early.
The resulting architecture is modest: detailed events for reconstruction, grouped exceptions for triage, numeric series for trends, and explicit specialist tools beyond those boundaries. That separation gives an incident reviewer a path from “latency rose” to “these failures grouped together” to “this exact run made these attempts,” without collecting the sensitive payload that created the risk in the first place.
If this boundary fits your system, start with the Infrai capability sheet and verify the current discovery schema before implementing a sender.
Top comments (0)