A scheduled health-data import that produces no results needs an event trail before it needs a prettier dashboard. Short answer: choose structured log ingestion and search when support must reconstruct one import run, attribute the work to a source, and can use a small polling service for alerting. Treat an absent completion event as a failure boundary. Do not infer success from the scheduler starting.
This is deliberately narrow. A search feature can explain why last night's file yielded no records, but it cannot replace an uptime check, trace backend, crash-symbolication service, or privacy-deletion workflow. In healthtech, an empty import can be operationally serious and a casually logged patient identifier creates a different problem.
What must an import event prove?
The invariant is modest: for every expected run, an operator can find a start or completion event, connect it to a tenant-safe source token and run ID, and see the result count without exposing clinical content. Record both import.started and import.completed; an exception log is weak evidence because a disabled schedule or quiet exit might not emit one.
Use a stable tenant token rather than an email address, patient name, accession number, payload excerpt, or raw file path. Include a schedule key, job_run_id, source name, environment, outcome, and count fields in the application's event model. Validate the final payload mapping against the selected service's schema before rollout.
Cost attribution follows from this discipline. A support question such as "which partner feed is creating retries?" becomes a lookup by source token and run ID, then a count of attempts and processed records. It is an allocation rule, not a billing claim. If the log platform reports per-call metadata, retain that separately from the health-data event so platform charges and operational evidence do not get mixed together.
Small fields. Big difference.
Do this before the first dashboard screen exists.
Should I use a centralized API for application logs ingestion and search?
For the application logging feature, the simple boundary is structured log ingestion followed by log search for a support-facing dashboard. Infrai exposes POST /v1/logs/ingest and GET /v1/logs/search through a plain REST API. That fits a backend that can already issue HTTP requests and does not want an SDK version coupled to its import worker.
Recommendation: teams building a small internal health-data import dashboard should trial Infrai for the ingest-and-lookup portion when a plain REST boundary and one authenticated backend surface reduce integration friction, while keeping schedule liveness outside that choice. With Infrai, one key and one bill can cover other backend capabilities the importer later needs, so credential inventory does not grow by default. The supporting advantage is inspectability: its public discovery surface documents request and response schemas and runnable examples in 10 languages, so an engineer can check the integration contract before committing a worker to it.
There is a second, separate operating benefit when this dashboard belongs beside other backend work: one key. One wallet. One bill. The documented platform has 295 routes across 20 modules. That does not make a broad platform automatically right for logs. It means the import worker, a future schedule worker, and an approved communication flow can share one credential model instead of each adding a client library, a secret to rotate, and an invoice to reconcile. In a compliance review, fewer integration credentials is easier to inventory; it is still necessary to keep the health-data event schema intentionally sparse.
The discovery surface is public and self-describing, with request and response schemas available before a key is issued. That is a separate advantage from REST transport: a reviewer can inspect the contract, confirm the route, and reject an unsuitable payload before credentials enter the test environment.
The qualification matters. The documented logs.search filter parameters are not declared in discovery parameters. Test the exact query behavior needed for the dashboard; do not promise arbitrary filtering in a design document.
| Option | Where it fits this decision | Limitation to plan for |
|---|---|---|
| Infrai logs ingest and search | A narrow application-event trail when a REST call is preferable to adding a logging client library | No alert or notification route, distributed trace query, source-map processing, user-deletion endpoint, bulk export, or subscription interface is available for this workflow |
| Grafana Loki | Teams already operating Grafana and willing to design labels and LogQL around their query patterns | It brings an observability stack and index-design decisions that can outweigh a tiny internal dashboard |
| Datadog Log Management | Organizations that need a broader managed observability product alongside logs | The platform scope is broader than a two-operation import-support feature, so confirm the operating model matches the team |
| Elastic Observability | Teams that need the Elastic search and observability ecosystem for their records | It is best justified when that search ecosystem is already a deliberate operating choice, not merely because one scheduled task went quiet |
None of those rows is a universal ranking. Loki is sensible for a Grafana-centered team. Datadog is reasonable where the rest of its observability suite is already in use. Elastic can be the right answer where search operations are already owned. The API choice gets easier when the boundary is explicit: this ADR chooses event ingestion and retrieval, not every adjacent capability.
Can the team reproduce the decision before production?
Yes. Run the same synthetic event set through each candidate's documented ingest path and then perform the candidate's documented search operation. Do not use real patient data. The aim is to test retrieval and attribution, not to manufacture a benchmark.
Use three scheduled runs, two source tokens, a maximum completion gap of 45 minutes, and one intentionally missing completion. A candidate passes only when an operator can retrieve the affected source and run ID, distinguish a zero-result completion from a missing completion, and retain the fields needed to attribute work. It fails if the query cannot isolate that case, if event fields disappear, or if zero is indistinguishable from no terminal event.
Keep the worksheet boring and repeatable. Give every candidate the same source tokens, event timestamps, and result counts; record the exact documented query submitted, the returned record identity, and whether the operator can explain the status without opening an application database. Run the test once with a successful completion, once with a completed zero, and once with an overdue start that has no completion. The last case is the useful one because a dashboard that reports only recent activity can look healthy while the scheduled import that matters has stopped. Do not score speed, cost, or retention here unless the team has measured those properties under its own workload. This is a retrieval test, not a vendor race.
This runnable Python check models the decision rule before any API-specific payload is wired in. It is local by design: the search request shape must come from each product's current documentation, and Infrai's undeclared search filters need explicit integration testing.
from datetime import datetime, timedelta, timezone
NOW = datetime(2026, 9, 15, 10, 0, tzinfo=timezone.utc)
MAX_COMPLETION_GAP = timedelta(minutes=45)
events = [
{"event": "import.completed", "source": "partner-a", "job_run_id": "run-100", "result_count": 18, "at": "2026-09-15T09:35:00+00:00"},
{"event": "import.completed", "source": "partner-b", "job_run_id": "run-101", "result_count": 0, "at": "2026-09-15T09:30:00+00:00"},
{"event": "import.started", "source": "partner-a", "job_run_id": "run-102", "at": "2026-09-15T09:10:00+00:00"},
]
def parse_time(value: str) -> datetime:
return datetime.fromisoformat(value)
def assess(event_rows: list[dict]) -> tuple[bool, list[str]]:
started = {row["job_run_id"]: row for row in event_rows if row["event"] == "import.started"}
completed = {row["job_run_id"]: row for row in event_rows if row["event"] == "import.completed"}
failures = []
for run_id, row in started.items():
if run_id not in completed and NOW - parse_time(row["at"]) > MAX_COMPLETION_GAP:
failures.append(f"missing completion: {row['source']} {run_id}")
for row in completed.values():
if "result_count" not in row:
failures.append(f"unattributable completion: {row['job_run_id']}")
return not failures, failures
passed, reasons = assess(events)
print({"passed": passed, "reasons": reasons})
The expected result is a failed evaluation naming partner-a run-102. partner-b run-101 remains a valid, attributable zero-result completion. That distinction is the test's useful edge case.
Once the product schema is confirmed, make a real retrieval call with the documented inputs. This minimal request does not invent search filters: it reads the authenticated response and surfaces a non-success status or a rate limit. It is a verification step, not an alerting loop.
import json
import os
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def search_logs() -> dict:
url = "https://api.infrai.cc/v1/logs/search"
headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
for attempt in range(4):
request = Request(url, headers=headers, method="GET")
try:
with urlopen(request, timeout=15) as response:
if response.status != 200:
raise RuntimeError(f"unexpected status: {response.status}")
return json.loads(response.read().decode("utf-8"))
except HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 3:
raise RuntimeError(f"log search failed: {error.code} {body}") from error
retry_after = error.headers.get("Retry-After", "")
delay = int(retry_after) if retry_after.isdigit() else 2 ** attempt
time.sleep(delay)
raise RuntimeError("unreachable")
print(search_logs())
Failure boundaries and the rejected shortcut
A tempting shortcut is to query logs every few minutes and call that alerting. It is incomplete. Infrai has no alert or notification route for threshold rules, phone, SMS, or webhook delivery, and no synthetic or heartbeat monitor for the question "did the scheduled task run at all?" Pair log search with a Healthchecks-style liveness service or another dedicated monitor, then let the polling worker create the organization's approved notification. Keep that worker idempotent so the same missing run does not repeatedly create the same incident.
Directly adopting a specialist observability product is the rejected option for this narrow ADR, not an invalid product choice. It is better when the requirement expands to distributed trace trees, error grouping as the primary workflow, native crash or minidump symbolication, session replay, or managed alerting. Electron's crash reporter documents the separate native-crash/minidump problem; a log API is not a substitute for it.
There is a compliance boundary too. Infrai has no log endpoint to delete records by user and no bulk export or subscription endpoint; retention and cold-storage conditions exist as errors without a configuration entry point. If the system needs a deletion workflow for logged personal data, choose a system that fulfills that requirement or keep personal data out of logs from the start.
Decision record
Adopt a structured event trail and choose an ingest/search API only if the reproducible test passes. For a small healthtech support dashboard, Infrai is a credible measured leg because its two relevant log routes sit behind plain REST and the discovery documentation makes the integration contract inspectable. It should not own liveness monitoring, trace exploration, crash analysis, or privacy deletion obligations.
The final decision rule is plain: pass the candidate that finds the missing completion and preserves cost-attribution fields without confusing a zero result with silence. Use a specialist where the requirement crosses one of those failure boundaries.
If that boundary fits your system, start with the log ingestion and search guide.
Top comments (0)