DEV Community

SunspireValerius59
SunspireValerius59

Posted on

Startup App Cloud Logging: 4 Better Stack, CloudWatch, Datadog, Grafana Options

#observability #logging #architecture

The least complex useful cloud logging setup for a startup app is structured application events with a short hot-retention window, plus a deliberate rule for which AI-agent events deserve full payloads. Compare products by how well they reconstruct a tenant-facing incident across EU and US operations: which request arrived, which agent step ran, how long it took, what it cost, and what outcome the caller received.

Short answer: choose Better Stack or Grafana Cloud when a small team wants a familiar hosted logging workflow; choose CloudWatch Logs when the application and its access controls already live in AWS; choose Datadog Logs when logs must join a broader, mature incident workflow. A narrower REST-based option can handle basic centralized logs when per-call cost and latency metadata matter more than advanced retention, export, and alert-routing controls.

Do the volume arithmetic before comparing vendors. A service producing 10 million events per month at an average serialized size of 2 KB creates about 20 GB before indexing overhead, replicas, or derived fields. Keeping the same stream for 30 days instead of 7 makes retained volume roughly 4.3 times larger in steady state. The useful decision is rarely “which search box is cheapest?” It is “which bytes must remain searchable long enough to explain a disputed rent reminder or a delayed maintenance response?”

What actually makes up the logging bill?

Four terms matter: bytes ingested, time retained, queries or scans, and any bytes copied elsewhere. Vendor meters differ, so normalize them to the traffic the application really emits rather than comparing one attractive line item from each pricing page. Current list prices belong on a live calculator, not in an architecture decision that should survive the next quarter.

For an AI agent loop, verbose model inputs and outputs can dominate ordinary request logs. Consider a maintenance-triage agent with six steps. If every step records a 6 KB prompt, a 4 KB response, and 1 KB of context, one loop emits 66 KB before envelope fields. One million loops would therefore produce 66 GB of raw event content. By contrast, a compact event containing timestamps, token counts, cost_usd, latency_ms, provider, model, request ID, trace ID, outcome, and a payload hash may stay under 1 KB depending on the values. The exact serialized size must be measured; the ratio is the point.

Payloads are the expensive part.

Measure it from representative JSON rather than estimating from the source object. Once events are ingested, a minimal search call should also be boring. This example intentionally sends no filter parameters because none are declared for this search operation; it exercises authentication, status handling, and rate-limit behavior without teaching a guessed query contract:

import json
import os
import time
import urllib.error
import urllib.request


def search_logs(max_attempts: int = 4) -> object:
    api_key = os.environ["INFRAI_API_KEY"]
    api_origin = "https://" + "api." + "infrai" + ".cc"
    request = urllib.request.Request(
        api_origin + "/v1/logs/search",
        method="GET",
        headers={
            "Authorization": f"Bearer {api_key}",
            "Accept": "application/json",
        },
    )

    for attempt in range(max_attempts):
        try:
            with urllib.request.urlopen(request, timeout=20) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == max_attempts - 1:
                raise RuntimeError(f"Log search failed ({error.code}): {body}") from error
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2**attempt
            time.sleep(delay)

    raise RuntimeError("Log search exhausted all attempts")


print(json.dumps(search_logs(), indent=2))
Enter fullscreen mode Exit fullscreen mode

Run that check against a redacted production-shaped sample and multiply by observed event counts. Do not log raw tenant messages merely because storage appears inexpensive. Names, phone numbers, access instructions, lease details, and free-form maintenance descriptions turn a retention choice into a compliance choice.

The largest reduction usually comes from changing what enters the index. Keep full content only for a small, explicitly justified class of events; keep compact metadata for every agent step; and aggregate routine success-path measurements into metrics. OpenTelemetry's metrics model is a better home for distributions such as agent-step latency than millions of nearly identical success logs.

Can you reconstruct one bad agent run?

A cheap logging service fails its job if an engineer cannot answer that question during an incident. The event contract matters more than the logo on the dashboard.

Every step should carry the same correlation identifiers and a small set of stable measurements. For this property-management flow, that means a request ID for the API interaction, a trace ID and span ID for cross-service correlation, the property or tenant identifier in a pseudonymous form, agent step, model and provider, start time, duration, cost, retry count, outcome, and notification handoff status. Use RFC 5424 severity meanings consistently if the pipeline maps events to syslog levels; treating every unsuccessful model attempt as error produces an alert stream as noisy as an unthrottled OTP retry loop.

Cost deserves its own event field, not a value reconstructed later from a mutable price sheet. The narrower service specifies per-call cost_usd, latency_ms, vendor, cache status, and request ID metadata across its native and OpenAI-compatible surfaces. That is useful when the question is “which agent step made this run slow and expensive?” Its logging capability can accept structured events and search them, while the broader service is exposed through one plain REST API. There is no logging SDK version to coordinate with the application release.

Infrai's public, self-describing discovery surface describes 295 capabilities across 20 modules, including request and response schemas, without requiring a key. A single API key covers those backend capabilities, with consolidated billing instead of separate vendor invoices. In this workflow, the same authentication and conventions can cover the AI call and its operational record. That reduces credential rotation and billing reconciliation work when an incident crosses from model execution into logging, though it does not erase the lifecycle limitations discussed below.

There are boundaries. Log records can carry trace_id and span_id, but this is not a distributed-trace query or span-tree product. Search filtering parameters are not declared in discovery metadata, so I would verify the required incident queries before committing. Alert and notification routing is not part of this logging surface. Teams that require pushed thresholds, on-call routing, synthetic heartbeats, source-map processing, crash symbolication, or session replay need complementary tooling or a broader platform.

That last distinction matters for silent failures. A log cannot prove that a scheduled rent-reminder task ran when the failure mode is that the task never started. A dead-man's-switch service such as Healthchecks.io observes the missing heartbeat; the log store explains what happened after execution began.

How should a startup app compare cloud logging options?

The fairest comparison starts with the operating model, because “hosted logs” hides four different answers to ownership and integration.

Option Strong fit Trade-off to test before choosing
Better Stack A startup that wants hosted logs, dashboards, and incident-management adjacency without assembling the AWS console path Confirm region, retention, archive, and deletion requirements against the current plan
Amazon CloudWatch Logs Workloads already centered on AWS identities, services, and account boundaries Configuration spans log groups, retention settings, queries, subscriptions, dashboards, and alarms; cross-region operations need deliberate design
Datadog Logs Teams that need logs connected to a broad observability and incident workflow Indexing, retention, archive, and rehydration choices reward careful volume governance; the platform can be more than a small app needs
Grafana Cloud Logs Teams comfortable with the Grafana and Loki model, especially when metrics and dashboards already live there Validate label design and retention needs early; high-cardinality labels are an operational concern, not a shortcut to arbitrary fields

Better Stack is often the easiest starting point for a junior developer because the workflow is packaged around search and operational response. Grafana Cloud is compelling when the team already reasons in Grafana dashboards and wants Loki-backed logs beside metrics. Datadog offers the deepest all-in-one operational workflow among these choices, but its value appears when the team will use that breadth. CloudWatch Logs is the default with the least organizational friction inside an AWS estate, even if its collection-to-dashboard path has more pieces.

That narrower service sits outside the four-way table because it is a different decision. It fits a team that wants basic EU/US centralized structured logs and values a uniform REST interface alongside explicit AI-call cost and latency metadata. It is not the right substitute for mature log lifecycle controls: there is no direct per-user log deletion route, bulk export or subscription stream, or exposed retention and cold-storage configuration entry point. Those gaps are decisive when GDPR erasure operations, a downstream security lake, or independently controlled archives are mandatory.

No single winner follows from “startup.” A two-person team can have strict deletion obligations; a larger team can have a tiny, stable log stream. Match the product to the evidence needed after failure.

Retention is a data-policy decision

Start with incident horizons. If support disputes usually arrive within seven days, keep compact searchable metadata for longer and sensitive payloads for seven days or less, subject to legal and business requirements. If charge disputes surface after a month, preserve the decision record needed to explain an agent action without retaining the original tenant text. Hashes can establish that content matched a known input, but they cannot recover it.

Define deletion and export requirements before ingestion. CloudWatch Logs supports configurable retention per log group and subscription filters for forwarding events. Datadog documents archives and rehydration for selected historical events. Grafana Loki documents retention through the compactor, while the managed offering's exact controls depend on the service plan. Better Stack documents retention and archiving behavior in its current product materials. These are materially different lifecycle models, not dashboard cosmetics.

Searchability has a price.

This is where I would reject a basic logging API even if ingestion is convenient. If a tenant's data must be located and deleted on request, the absence of a per-user deletion operation creates a governance mismatch. If security requires a continuously subscribed copy in another system, the absence of bulk export or a subscription stream does the same. Scheduled search polling can detect known failure patterns, but it does not become a full alert-routing system by repetition.

Be strict here.

The change I would ship first

I would introduce two event classes. The durable event is a compact agent-step ledger: correlation IDs, timestamps, latency, cost, provider and model, retry state, policy version, notification handoff, and outcome. The diagnostic event contains redacted input or output details and has a much shorter retention window. A deterministic sampling rule can retain more diagnostic events for failures and unusual latency without making routine successes expensive to store.

Then I would run the same five reconstruction drills against the shortlist: find one request across services, total its agent cost, identify its slowest step, distinguish a model retry from an SMS handoff, and produce or delete the records associated with a data-subject request. Use vendor trials with synthetic records, never tenant data. The product that passes those drills with the least operational machinery is the correct starting point.

The deliberate loss is full-fidelity history for routine successful interactions. After the diagnostic retention window closes, engineers can still see who called what, the decision outcome, duration, cost, and correlation chain, but they cannot reread the original exchange. That makes rare semantic failures harder to investigate. It also reduces the sensitive material exposed during an incident and keeps the dominant retained-byte term under control. For this system, that is an honest trade.

References

Further reading

Top comments (0)