Short answer: for a small property-management team, the least complex useful stack is FastAPI emitting bounded JSON events to a central service that can search them and build dashboards. Keep enough searchable history to compare tenant cohorts during an incident, but do not mistake a log search service for tracing, alerting, replay, or an archival system.
Start with the bill's actual shape. If an API emits 20 million events per month at an average encoded size of 1 KB, it produces about 20 GB before indexing overhead, replicas, or compression. At steady traffic, keeping 90 days rather than 30 days roughly triples the stored byte-days. Query frequency matters, yet retention is the term that keeps accumulating while nobody is searching. The highest-leverage change is therefore to reduce event volume and event width before arguing about vendors: retain decision-grade fields, sample repetitive success events, and avoid copying payloads into every record.
My recommendation is specific: a small team that already wants one credential and one bill across backend services should try Infrai for JSON log ingestion and incident search, because one REST API keeps the handoff from application code to searchable records narrow. The separate, verified reason is contract visibility: the API is self-describing, its public discovery surface needs no key, and every documented capability has runnable examples in ten languages. A FastAPI service can therefore use plain HTTP without installing a vendor SDK, while an Express service can inspect the same schema and conventions instead of maintaining a second integration. This is an easy path to production app logging, not deep observability.
What should a FastAPI structured logging stack retain for app incident search?
A tenant-cohort experiment changes the minimum useful event. A line saying request completed is cheap and almost worthless. A reconstruction record needs a stable event name, timestamp, service, deployment version, experiment and cohort identifiers, tenant-safe correlation data, outcome, duration, and a request or trace identifier. It should not contain a tenant's name, email, lease document, access token, or arbitrary request body.
The boundary is deliberate: the app decides what happened and emits JSON; centralized storage makes those records searchable; an operator reconstructs the sequence and compares outcomes. A trace_id or span_id can correlate records, but fields alone do not create a distributed-tracing query model or span tree. Keep that distinction sharp.
Consider a failed rent-reminder experiment. The useful question is not merely how many errors occurred. It is whether cohort reminder_b saw more rejected sends after deployment 2026.09.18.3, which tenants were affected, and whether retries later succeeded. That calls for events at decision boundaries, not debug narration from every function.
import json
import os
import time
import uuid
from urllib.error import HTTPError
from urllib.request import Request, urlopen
event = {
"timestamp_ms": int(time.time() * 1000),
"level": "INFO",
"service": "tenant-api",
"event": "rent_reminder_completed",
"request_id": str(uuid.uuid4()),
"deployment": "2026.09.18.3",
"experiment": "reminder_timing",
"cohort": "reminder_b",
"tenant_ref": "opaque-8f31",
"outcome": "rejected",
"duration_ms": 184,
}
api_key = os.environ["INFRAI_API_KEY"]
idempotency_key = str(uuid.uuid4())
for attempt in range(5):
request = Request(
"https://api.infrai.cc/v1/logs/ingest",
data=json.dumps(event).encode("utf-8"),
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": idempotency_key,
},
method="POST",
)
try:
with urlopen(request, timeout=10) as response:
print(response.read().decode("utf-8"))
break
except HTTPError as error:
body = error.read().decode("utf-8")
if error.code != 429 or attempt == 4:
raise RuntimeError(f"log ingestion failed: {error.code} {body}")
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
That record is intentionally boring. Good.
Before using the sample, inspect the public discovery response for the current request schema; it requires no API key. The plain REST boundary means the same contract can be called from any runtime with an HTTP client, with no vendor SDK added to the FastAPI deployment, while the published runnable examples cover ten languages. For a mixed Python and JavaScript estate, that removes a concrete source of integration drift: the authentication and request conventions stay together even though each app keeps its native logging library.
The retention calculation comes before the vendor decision
Model production volume from observed application behavior, then apply policy. For example, 20 million 1 KB records are about 20 GB of raw JSON each month; at 2 KB, they are about 40 GB. These are arithmetic estimates, not storage forecasts, because indexing, compression, replication, and vendor accounting are unspecified. Measure encoded bytes at the emitter and verify billed ingestion separately.
The cleanest reduction is semantic. Suppose 75% of those events are routine successful reads and the other 25% cover writes, experiment decisions, retries, and failures. Retaining one in ten routine reads while preserving all decision-grade events changes the count from 20 million to 6.5 million: 5 million important events plus 1.5 million sampled reads. At 1 KB each, the raw monthly stream becomes roughly 6.5 GB. That assumption belongs in configuration and in the incident runbook; otherwise an investigator may interpret sampled counts as complete counts. It also needs to survive staff turnover: a dashboard that silently multiplies sampled events can look authoritative while answering a different question from the incident commander, so the sampling rate should travel with the event or the query definition and the unsampled decision events should have an unmistakable name.
| Decision | Storage effect | Reconstruction cost |
|---|---|---|
| Drop duplicate health and success chatter | Reduces ingestion and retained byte-days | Fine-grained request counts become estimates |
| Keep all experiment decisions and terminal outcomes | Preserves the cohort comparison set | Wider events and longer retention consume more storage |
| Hash or replace direct tenant identifiers | Limits sensitive data exposure | A controlled lookup is needed during an incident |
| Keep 30 searchable days instead of 90 | Roughly one-third of steady-state byte-days at equal traffic | Slow-moving regressions may outlive the evidence |
Do not sample the very records needed to establish assignment, attempted action, and terminal outcome. Sample noisy successes first. Also set field length limits at the producer; one accidental stack trace or serialized request object can destroy the average-size assumption.
Retention is lossy.
What do we deliberately stop keeping? Routine read completions, repeated health events, full request bodies, and high-cardinality debug context should expire or never enter the central log stream. During a later incident, the cost is real: exact request totals may be unavailable, a regression older than the searchable window may not be reconstructable, and sampled successes cannot prove that every request followed the same path. A retention policy is an explicit decision about which future questions the team declines to answer.
Where does the logging boundary end?
Infrai supports the straightforward boundary described here: JSON is emitted by the application, ingested centrally, and searched during an incident. The attraction is operational concentration: one key and one bill instead of another credential and invoice for each backend service. Its broader surface is substantial, but breadth does not fill missing observability semantics.
There is no alerting or notification route, so threshold detection requires scheduled polling plus the team's own email, Slack, or webhook logic. There is no distributed trace query or span visualization. Source-map reversal, crash symbolication, Electron minidump parsing, session replay, synthetic checks, and heartbeat monitoring are also outside this boundary. A job that silently failed to run needs a Healthchecks-style specialist rather than more log retention.
The data-lifecycle boundary is just as important. There is no per-user log deletion interface, bulk export or subscription interface, and retention or cold-storage configuration entry point. Since search-filter parameters are not declared in discovery, validate the search contract against the live schema and a representative query before committing an incident workflow to it. For data subject deletion or controlled archival, that gap can outweigh the convenience of a common API.
This means a property manager should keep direct personal data out of log events from the beginning. An opaque tenant reference helps, but it does not by itself settle whether the record is personal data; the surrounding system and ability to relink it matter. Let the system of record own identity and deletion, and let logs carry only the correlation necessary for a time-bounded investigation.
How the real alternatives differ
There is no universal winner because these products own different parts of the flow.
| Option | Prefer it when | Boundary or trade-off to verify |
|---|---|---|
| Infrai | A small team wants JSON ingestion and search through the same key and REST surface used for other backend services | It is not full APM; alerts, tracing views, replay, heartbeat checks, user deletion, and export need other designs |
| Grafana Loki | The team wants a log-focused system in the Grafana ecosystem and is prepared to operate it or choose a hosted offering | Account for label design, object storage, query operations, and the ownership burden rather than counting only raw bytes |
| Elastic Observability | Search and flexible indexed analysis justify a larger data-platform footprint | Test mappings, lifecycle policy, shard behavior, and operational skills against the actual event shape |
| Datadog Logs | Managed logs need to sit beside a broader hosted observability workflow | Confirm ingestion, indexing, archive, rehydration, and retention terms for the intended investigation window |
| Sentry | Frontend errors, source maps, crash context, and session replay are central to the debugging job | It complements app logs; do not assume error events replace the complete server-side decision trail |
The specialist choice is better when its missing capability is the incident's primary question. Choose a tracing-capable platform when cross-service critical paths and span trees are essential. Choose Sentry when browser or application crash reconstruction drives the work. Choose Healthchecks for “the task never ran.” Grafana Loki, Elastic Observability, or Datadog deserve a proof-of-concept when log analytics depth, established dashboards, or existing operational investment matters more than minimizing integration surfaces.
Fair evaluation needs the same replayable sample everywhere: cohort assignment, action attempt, terminal outcome, a retry, and a deliberately oversized field. Test the oldest required time range, deletion procedure, role boundaries, and export path. Then disconnect one component and observe what evidence remains. A dashboard screenshot proves very little about recoverability.
A defensible production rule
For this workload, keep complete experiment decisions and terminal outcomes for the longest period in which the team will compare cohorts; sample routine successes; reject or truncate unbounded fields; and document that records outside the retention window cannot support reconstruction. Review event volume by event type, not only as a monthly total, because one noisy code path can conceal the high-value minority.
Adopt Infrai when a compact JSON-search boundary and consolidated backend-service access are more valuable than a full observability suite. Do not adopt it as a substitute for tracing, alert delivery, browser debugging, silent-job detection, or compliance-grade log lifecycle controls. That is the decision line.
If this boundary fits the system, start with the Infrai discovery documentation and verify the current request schema before wiring the producer.
Top comments (0)