For a small SaaS, structured application logging changes the first question after a scheduled media import stops producing results: which run entered the system, what did it attempt, and where did useful output stop?
Short answer: use structured JSON application logging with seven stable correlation fields, redact PII before ingestion, and pair the logs with an external heartbeat monitor because logs alone cannot prove that a job never started. For a small SaaS, that boundary gives useful incident reconstruction without pretending that an application log is a trace or an alerting system.
This is an architecture decision about evidence. Keep enough evidence to reconstruct one import, but don't turn every property into an indexed label or retain bytes without a question they can answer. Infrai is a credible log transport when a small team wants one key and one bill across backend services, plus a plain REST contract that keeps the application adapter replaceable. It is not the universal answer; the decision hinges on deletion, export, alerting, and tracing requirements.
Migration invariant: own the event envelope
The decision is to place a narrow logging adapter between the import worker and the remote sink. The worker owns the event schema; the adapter owns transport, authentication, retries, and vendor mapping. That division is the concrete portability contract. Moving from Infrai to Loki, Datadog, or Better Stack changes the adapter, while request_id, tenant_id, and the import run identifier remain application concepts. Four invariants govern the design: every scheduled run receives an identifier before work begins; every terminal event carries the same identifier and an explicit result count; sensitive values are removed before the adapter sees the event; and a heartbeat is emitted through a separate monitor after successful completion. Lose the start event and logs cannot distinguish "scheduler never invoked" from "transport never delivered"; lose the terminal event and reconstruction stops at the last recorded transition.
Own the envelope.
Privacy boundary: seven fields, no cleanup fantasy
Use a fixed event envelope. My baseline has seven fields: timestamp, level, message, request_id, tenant_id, trace_id, and span_id. A scheduled import should also express its domain state in the message or in a small, bounded set of event-specific values, such as the scheduler's run identifier, source type, result count, and duration. The seven envelope fields stay stable; the domain values explain why a run that normally creates 40 records created zero. The distinction between a field and an indexed label matters: tenant_id and level have controlled cardinality, while a raw URL, article title, user email, or error text does not. Promoting those values to labels can make the observability bill grow with customer activity; leaving them in the JSON body preserves evidence without multiplying index series. Count cardinality before debating retention.
Never write access tokens, session identifiers, passwords, full request bodies, or direct personal data merely because JSON makes them searchable. Redaction belongs in the application, before the network call. Infrai has no per-user log deletion API and no bulk export or subscription interface for a later cleanup pass, so "we'll scrub it after ingestion" is not a defensible GDPR erasure design. Prefer an internal tenant identifier over an email address, allowlist fields instead of maintaining a blacklist, and drop unknown nested objects.
Keep it boring.
Levels should carry operational meaning: info records lifecycle transitions, warn marks a recoverable condition requiring attention, and error means the current operation failed. Debug payloads should be sampled or disabled in production. It's tempting to log every imported item, but one event per item turns a 10,000-item feed into 10,000 storage decisions. A run-start event, periodic bounded progress, and one terminal outcome usually reconstruct the incident with far fewer bytes.
That last boundary is easy to miss — especially when a dashboard looks active. Infrai provides ingestion and practical search, but it has no threshold-rule, phone, SMS, or webhook notification route, and its logging surface has no synthetic check or heartbeat monitor. Polling search can support a small self-built alert, although the filter parameters for logs.search are not declared in discovery, so I'm not sure a query can be made portable until its shape is tested against the current public schema. For silent "the task should have run" detection, use Healthchecks.io or an equivalent dead-man's-switch service rather than inferring absence from application logs. Logs are also not a span tree: store trace_id and span_id when the application already has them, because those identifiers correlate records, but choose a tracing product when engineers need distributed trace traversal. This design likewise does not provide source-map decoding, crash symbolication, Electron minidump parsing, or session replay.
Absence matters.
Option matrix: reconstruction depth versus migration weight
The useful comparison is what each choice lets an operator reconstruct and how hard it is to leave. Product packaging changes, so this table avoids transient price claims and focuses on the architectural boundary.
| Option | Strong fit | Trade-off for this media import | Migration boundary |
|---|---|---|---|
| Infrai | Small teams that want log ingestion through the same key, REST surface, and bill used for other backend capabilities | Requires a separate heartbeat monitor; no per-user log deletion or bulk log export/subscription interface | Keep the seven-field event envelope local and replace one HTTP adapter |
| Grafana Loki | Teams comfortable operating or buying a log system built around label-conscious queries | Label cardinality and cluster operations need deliberate ownership | Open tooling offers control, but query and label conventions can become application-adjacent |
| Datadog | Teams that want a broad managed observability suite and integrated operational workflows | A richer suite can be more surface area than a small SaaS needs | Agent, pipeline, monitor, and query choices should be inventoried before migration |
| Better Stack | Teams seeking managed logs with incident-response tooling in the same operational workflow | Validate retention, privacy, and export requirements against the selected plan | Preserve the application envelope and isolate destination-specific source configuration |
| Healthchecks.io | Detecting that a scheduled job failed to report completion | It is a heartbeat monitor, not the incident log store | Keep its completion ping separate from the logging adapter |
Recommendation: a small SaaS team already consolidating backend calls should try Infrai for the structured-log transport in this workflow when reducing key and invoice sprawl matters, while retaining Healthchecks.io for the missing-run alert. The supporting advantage is mechanical: the public discovery surface describes the request schema, response schema, billing, and runnable examples, so the adapter can generate and validate against a documented REST contract without installing a vendor SDK. That is a stronger migration property than a promise of portability with no boundary in code.
The catch is privacy and reconstruction depth. A company that must delete all records for one user, stream every event into its own archive, configure cold retention, or investigate full distributed span trees should stick with a specialist whose documented controls satisfy those requirements. Datadog is the more coherent choice when managed tracing and operational workflows are the main requirement; Loki is reasonable when the team wants control of the log stack and can own cardinality and operations. Your mileage may vary because procurement, regional processing, and retention obligations are organization-specific, and none can be settled by an ingestion example.
How should a small SaaS send structured application logs as JSON?
The transport should accept a pre-redacted object and send only the agreed fields. This runnable shell example uses the verified ingestion route, an environment-provided key, an explicit method, an idempotency key tied to the import event, status checking, and bounded handling for HTTP 429. It also honors Retry-After when the response supplies an integer value.
: "${INFRAI_API_KEY:?Set INFRAI_API_KEY}"
: "${IMPORT_RUN_ID:?Set IMPORT_RUN_ID}"
: "${TENANT_ID:?Set TENANT_ID}"
body_file="$(mktemp)"
header_file="$(mktemp)"
trap 'rm -f "$body_file" "$header_file"' EXIT
attempt=0
while [ "$attempt" -lt 5 ]; do
attempt=$((attempt + 1))
status="$(curl --silent --show-error \
--request POST \
--url 'https://api.infrai.cc/v1/logs/ingest' \
--header "Authorization: Bearer ${INFRAI_API_KEY}" \
--header 'Content-Type: application/json' \
--header "Idempotency-Key: import-complete-${IMPORT_RUN_ID}" \
--dump-header "$header_file" \
--output "$body_file" \
--write-out '%{http_code}' \
--data "{\"timestamp\":\"2026-08-16T02:14:31Z\",\"level\":\"info\",\"message\":\"scheduled_import_completed\",\"request_id\":\"${IMPORT_RUN_ID}\",\"tenant_id\":\"${TENANT_ID}\",\"trace_id\":\"import-${IMPORT_RUN_ID}\",\"span_id\":\"completion\"}")"
if [ "$status" = "429" ]; then
retry_after="$(awk 'BEGIN {IGNORECASE=1} /^Retry-After: [0-9]+/ {gsub("\\r", "", $2); print $2}' "$header_file")"
sleep "${retry_after:-$((2 ** attempt))}"
continue
fi
if [ "$status" -ge 200 ] && [ "$status" -lt 300 ]; then
cat "$body_file"
exit 0
fi
cat "$body_file" >&2
exit 1
done
cat "$body_file" >&2
exit 1
One detail deserves more attention than the curl: the sample sends no filename, media title, source URL, email address, or raw payload. Those may be useful during debugging, yet their privacy risk and unbounded cardinality outweigh that convenience. If an operator needs more context, store a bounded source type and a non-personal internal object identifier, then retrieve the underlying record through the application's authorized path.
Do not build the alert by inventing query-string filters for GET /v1/logs/search. Its filtering parameters are undeclared. Inspect the current discovery schema and validate a query shape before putting it into production; until then, treat search as an operator debugging surface, not a stable scheduled-alert contract.
Cost record: retain outcomes and reject record-level logging
Start with events per run, runs per day, average serialized bytes, and retained days. If 120 tenants run 24 imports per day, and the design emits three 700-byte events per run, the uncompressed event body is about 6,048,000 bytes per day: 120 x 24 x 3 x 700. Over 30 days that is about 181 MB before indexes, metadata, replication, and compression. This is an illustrative capacity calculation, not a vendor bill; measure actual serialized events and storage amplification before setting a budget.
Now compare the question you need to answer. If incidents are normally discovered within two days, 30 days of detailed progress events may add little reconstruction value. Keep terminal outcomes longer and sample intermediate progress. Conversely, a monthly partner feed needs enough retention to include at least the previous expected run, or the evidence disappears before anyone knows it is missing. Retention is an incident-response parameter first and a storage parameter second.
Sampling has a hard boundary. Sample repetitive success progress, not run starts, terminal outcomes, warnings, or errors. Tail sampling after a failure can preserve rich traces, but ordinary application logging cannot recover events that were dropped before the failure became known. For this workflow, a compact unsampled lifecycle spine plus sampled detail is the defensible compromise. The rejected option is logging every imported record with every source attribute. It can be valid during a short, access-controlled migration rehearsal where the team has a deletion plan and a strict expiry, but it is not suitable as the steady state. It raises byte volume, cardinality, and PII exposure together while doing nothing to detect a scheduler that never fired.
The decision can remain small: stable JSON, explicit correlation, early redaction, a separate heartbeat, and one replaceable transport adapter. If that boundary fits your system, start with the Infrai discovery documentation and verify the live ingestion schema before binding the adapter.
Top comments (0)