Short answer: For a small marketplace SaaS, select a simple app logging service by proving that the same structured JSON evidence can leave it again: export one AI agent run, reconstruct the application-to-Postgres timeline from that export, and verify separate US and EU deletion. This exit-first test catches lock-in, broken causality, and hidden retention cost before setup convenience becomes commitment.
An agent loop may search inventory, call a model, invoke a tool, retry, write an offer, and answer the buyer. Search screens change. The evidence needed to explain that chain should not.
Migration design: How can a small SaaS move structured JSON app logs safely?
Define a portable incident record before sending production traffic anywhere. For each event, retain a timestamp, observed timestamp, severity, service name, deployment environment, region, agent run ID, trace ID, span ID, event name, duration, attempt number, and outcome. OpenTelemetry's log data model supplies useful semantics for timestamps, severity, body, resource, attributes, and trace context. It is a vocabulary here, not a product recommendation.
The export test is blunt: take one failed synthetic transaction out of the candidate system, load the records into a plain analysis environment, and recover the order of the model call, tool retry, Postgres operation, and final buyer response without a vendor-specific grouping rule. Preserve numeric durations as numbers and timestamps as timestamps. Preserve every attempt. If an export flattens types or omits correlation context, switching services later may preserve bytes while destroying the incident.
This is the first selection gate.
Error grouping and log reconstruction solve related but different problems. Sentry documents how fingerprints influence event grouping, which is useful when an exception family is the investigation unit. An agent incident may include no exception at all: a slow model response, a permitted retry, and a successful but late database write can still violate the marketplace promise. Stick with an error-focused workflow when grouped exceptions dominate support work. Use a log-oriented record when ordinary events must form a causal narrative.
Integration drill: replay the agent failure before connecting production
Create five synthetic runs with fixed outcomes: a normal purchase, a tool timeout followed by success, a Postgres lock wait, a policy denial, and a slow model response. Use invented account and order identifiers, with no personal data. For each run, write the expected event order and the questions an investigator must answer. Then send the same fixture to every candidate and give the replay task to an engineer who did not construct it.
The timeout case should be deliberately awkward. It needs an initial tool attempt, a timeout outcome, a second attempt, one database write, and a completion event. The investigator must distinguish application event time from telemetry arrival time, show which attempt produced the write, and establish total user-visible latency. If the log view silently replaces attempt one with attempt two, sorts solely by arrival, or requires a customer identifier that policy forbids storing, the design has failed its concrete job even if individual records look tidy. Record the query operations and elapsed investigation time, but don't present one fixture as a throughput benchmark.
Now disturb delivery. Submit malformed JSON and an oversized event at the generic ingestion boundary, interrupt network access, and exercise client backpressure. The application needs an explicit drop, bounded-buffer, and retry policy. Blocking marketplace checkout to protect a debug event is usually the wrong trade; the compact completion record should instead carry a telemetry-loss count so an investigator can tell missing evidence from missing work.
No dashboard can infer that distinction after the fact.
Governance boundary: keep detailed evidence regional and deletable
“Available in the US and EU” does not establish where detailed evidence travels. Document where ingestion terminates, indexed and archived copies reside, backups go, support access originates, and exports land. Use one schema with two destinations: EU application events go to the EU destination and US events to the US destination. Cross-region views can combine content-free counts and latency distributions, while detailed replay stays in the originating region.
An agent path may still cross a global model endpoint, a regional application process, and a Postgres primary in one jurisdiction. Logging cannot repair an invalid production data path. Map the path first, attach region and data-classification fields at event creation, and reject impossible combinations in deployment tests. An EU-classified synthetic account should never create a detailed diagnostic artifact in a US destination. The European Commission's standard contractual clauses are one legal mechanism for some transfers, but contracts do not replace minimization or an executed routing test.
Keep prompts, model responses, SQL text, access tokens, and raw user records outside routine logs. They are potentially sensitive and large. If a narrow investigation requires payload inspection, put a redacted artifact behind stricter access for a shorter period and log only its reference and deletion time. Search metadata first; open content only under the separate authorization path.
Separate destinations create duplicated retention settings, access reviews, deletion checks, and rollout verification — a real burden for a small team. They are not suitable when there is no regional data boundary and the team cannot keep configuration aligned. A single destination is operationally simpler, but it is not suitable when detailed records must remain region-bound. The system constraint decides.
Cost control: count bytes and cardinality only after successful replay
Once the fixture works, measure serialized bytes rather than estimating from log-line counts. Consider an illustrative marketplace processing 40,000 agent runs per day. A successful run emits 12 events averaging 900 bytes; 2% of runs fail and retain 30 additional diagnostic events averaging 1,400 bytes. Uncompressed volume is approximately 40,000 × 12 × 900 + 40,000 × 0.02 × 30 × 1,400, or 465.6 MB per day and about 14 GB over 30 days, before indexing overhead, replicas, compression, or protocol envelopes. This is retention math, not a benchmark. Replace every input with measurements from the fixture and trial.
| Record class | Trial policy | Reason |
|---|---|---|
| Failed-run metadata | Retain 30 days | Supports delayed incident reports |
| Successful-run metadata | Retain 7 days | Covers recent latency analysis |
| Redacted payload artifact | Retain 24 hours or less | Limits content exposure and bulk |
| Stable fields | Index 8–12 during the trial | Bounds index growth |
| High-cardinality IDs | Store as searchable values | Avoids using unique IDs as grouping labels |
Cardinality is often less visible than volume. region has a tiny domain; outcome may have four values; agent.run_id approaches one value per run; a raw URL or SQL statement can approach one per event. Count distinct values over an hour and a day, then mark each field indexed, stored-only, or excluded. Repeat after deployments. Replacing a route template with a full URL can turn a bounded field into an unbounded one without changing event count.
Sampling comes last because it changes the replay guarantee. Randomly retaining 10% of full runs can describe broad latency while discarding the only run linked to a buyer complaint. Keep a compact completion record for every run when measured size permits, sample verbose steps for normal successes, and retain detail for errors, timeouts, policy denials, and unusually slow runs. Tail decisions can use the known outcome; without them, the application can emit the compact record universally and richer events for defined failure classes. I'm not sure one percentage can cover both a quiet launch and a seasonal surge. Load tests using the measured size distribution resolve that uncertainty.
Your mileage may vary because services meter ingestion, retained or indexed bytes, query scans, exports, and network transfer differently. Normalize candidates to the same corpus, retention, and replay queries. “Cheap” has no useful meaning without those inputs and the engineering time required to operate the path.
Rollout decision: prove equivalence before cutting over one workflow
Shadow one marketplace flow to the new destination for a week. Compare completion-record counts with application request counts, replay all five fixtures in each region, inspect the largest records, and calculate distinct values for proposed indexes. Query results need not look identical, but both paths must answer the written incident questions from the same event contract.
Next, verify deletion after each configured window, including the documented behavior of archives and backups. Export a sample and confirm that field types and trace context survive. Switch the read path first, keep bounded dual delivery long enough to exercise real operational queries, and then remove the old delivery path. Dual writing forever doubles failure modes and weakens confidence about which copy is authoritative.
Expand one agent tool or Postgres workflow at a time. Treat schema changes like API changes, alert on measured volume and cardinality departures, and keep replay, export, regional routing, and deletion checks in deployment tests. A managed service is a reasonable fit when the team cannot operate storage and indexing; a self-hosted path can fit strict control requirements or existing platform expertise, provided upgrades, capacity planning, and on-call labor enter the decision.
The durable rule is short: make the incident portable, then decide where to keep it.
Top comments (0)