You can ask DataBuff about slow traces in plain English. That shipped in our last walkthrough.
Now ERROR logs join the party.
Same checkout demo, same InsufficientStockException on service-b — but this time we follow three real on-call paths: facet search in the Logs UI, trace-to-log deep links, and one-sentence AI queries that call log.queryLog* tools against live OTLP data.
All screenshots below are from a single incident window on a live demo environment.
Demo scenario: inventory checkout failure
The demo app hammers GET /demo/checkout. When stock runs out, service-b throws:
InsufficientStockException: inventory unavailable for skuId=…
OTLP logs land in Doris with trace_id and span_id attached. We walk the same failure three ways — the way a real shift would:
- Path A — You know the exception name; search the global Logs page
- Path B — You have a slow trace; read span logs and deep-link back
- Path C — You delegate to the AI squad in one sentence
Path A: global log search (no LogQL required)
Menu: Application Performance → Log Analysis
No query language required — keyword + facets is enough.
Steps: search InsufficientStockException → filter ERROR + service-b → 95 matching lines with an ERROR spike in the histogram.
Each row has Trace · View on the right — jump straight to the call chain without copying IDs.
Path B: trace ↔ log deep links
B1 · Trace header: one click to logs
Open a slow GET /demo/checkout trace (~240 ms). The trace header shows TraceID; Log Analysis on the right pre-fills traceId — no copy-paste.
B2 · Span sidebar: flame graph + Logs tab
Spans marked Logs on the flame tree open a sidebar Logs tab: a timeline from Received checkout request through Delegating inventory check to service-b. Select the service-b span to see the ERROR stack.
B3 · Deep link: "View all in Log Analysis"
Click View all in Log Analysis at the bottom of the sidebar. The global page auto-fills traceId + spanId and shows only the four logs in that span's context.
Path C: ask the AI squad about logs
The UI is for precision. The AI is for one-sentence delegation.
The Smart Query expert registers a log tool family — visible under Tool Management:
Registered tools:
-
log.queryLogDetail— search by service, level, keyword -
log.queryLogsByTraceId— all logs on a trace -
log.queryLogsBySpanId— logs for one span -
log.queryLogTrend— ERROR volume over time
Scenario 1: find ERROR logs by service + keyword
Find ERROR logs for service-b in the last hour related to
InsufficientStockException. List traceIds and key log summaries.
Scenario 2: known traceId → root cause
Given traceId edfa44615dcee4d6bdfeed46d84bfb20, list all ERROR-level
logs on this trace and explain why checkout failed.
The agent walks queryLogsByTraceId → 13-span call chain → ERROR lines → checkout failed due to insufficient inventory.
Scenario 3: ERROR volume trend
How has ERROR log volume for service-b looked in the last hour?
Any obvious spike periods?
Result: steady ~2 ERROR logs per minute — no spike, just a sustained inventory shortage.
Tool selection cheat sheet
- Search by service/level/keyword →
queryLogDetail - Known traceId →
queryLogsByTraceId - One span's context →
queryLogsBySpanId - Volume spikes →
queryLogTrend
Where the data comes from
OTLP Logs (:4317 / :4318) → Ingest → Doris log_dc_record → POST /log/search.
Inject traceId via Java MDC and logs correlate automatically.
DataBuff is log exploration in an APM context — not a replacement for ELK or Loki. The win is sharing context with traces, metrics, and AI agents without hopping three systems.
Try it (5 minutes)
curl -fsSL https://databuff.ai/databuff/ai-apm-install.sh | bash
curl -fsSL https://databuff.ai/databuff/ai-apm-demo-install.sh | bash
Open http://YOUR_HOST:27403 — login admin / Databuff@123 — add an LLM key under Settings → AI model.
Then try Path A, B, or C on the built-in checkout demo.
GitHub: https://github.com/databufflabs/databuff
OpenTelemetry Logs · Trace correlation · AI-native observability · Built in public
Top comments (0)