DEV Community

Cover image for Your Telemetry Has Data. Does It Have Context?

Your Telemetry Has Data. Does It Have Context?

Imagine you're investigating a slow checkout experience. Your application generates metrics, logs, and traces, but connecting the information across these signals isn't always straightforward.

For example, one component identifies a service as checkout-api, while another reports it as checkout. Although both refer to the same logical service, inconsistent naming makes it harder to connect related telemetry and understand what's happening.

This highlights an important challenge in distributed systems: collecting telemetry is only the first step. You also need consistent naming, shared context, and reliable ways to navigate between signals.

In the previous article, we explored how Prometheus, Jaeger, OpenSearch, and Perses fit into an open-source observability stack. Now, let's look at how to make telemetry from different services and signals work together.

We'll explore three concepts that make this possible: semantic conventions, resource attributes, and context propagation.

Semantic conventions

OpenTelemetry semantic conventions define standard names and meanings for telemetry attributes, events, and operations. They help instrumentation libraries and applications describe common concepts consistently, making telemetry easier for observability tools to interpret.

For example, OpenTelemetry defines standard HTTP attributes:

Attribute Example value What it describes
http.request.method POST The HTTP request method
http.route /payments/{payment_id} The matched route template
http.response.status_code 201 The HTTP response status code

However, semantic conventions don't enforce consistency automatically. Your instrumentation and telemetry pipeline must use and preserve the defined conventions. Custom attributes might still be necessary for application-specific details.

Note: Pay attention to attribute cardinality, especially when using attributes as metric dimensions. Use route templates such as /payments/{payment_id} rather than individual URLs such as /payments/12345. Recording unique values for every request can create high-cardinality metric data, increasing storage and query costs.

Resource attributes vs. span attributes

Resource attributes

A resource represents an entity that produces telemetry, such as a service, process, or container. Its attributes provide identifying information that can accompany the entity's metrics, logs, and traces.

For example, a payment-service instance might report the following resource:

{
  "service.name": "payment-service",
  "service.namespace": "checkout",
  "service.version": "2.4.1",
  "deployment.environment.name": "production",
  "service.instance.id": "payment-service-instance-01"
}
Enter fullscreen mode Exit fullscreen mode

Each attribute provides a different piece of information:

  • service.name identifies the logical service.
  • service.namespace groups related services.
  • service.version identifies the deployed version.
  • deployment.environment.name identifies the deployment environment.
  • service.instance.id distinguishes this instance from other instances of the same service.

For example, multiple instances of payment-service can share the same service.name, service.version, and environment while reporting different service.instance.id values.

Span attributes

A span represents an individual operation within a trace. Its attributes describe what happened during that operation.

For example, a span created when payment-service handles an HTTP request might contain:

{
  "http.request.method": "POST",
  "http.route": "/payments/{payment_id}",
  "http.response.status_code": 201
}
Enter fullscreen mode Exit fullscreen mode

These attributes describe the request handled by the service, rather than the service itself.

Note: This distinction helps you investigate telemetry at different levels. Resource attributes identify the source of telemetry, while span attributes provide details about individual operations.

Context propagation

A distributed request often passes through several services. To understand the complete request, you need to connect the spans generated by each service into one distributed trace.

OpenTelemetry context propagation carries information between services and execution units. For distributed tracing, this context includes identifiers such as the trace ID and span ID. The receiving service extracts the incoming context and uses it when creating its own span.

OpenTelemetry commonly uses the W3C Trace Context specification to propagate trace context across HTTP requests.

For HTTP requests, services commonly transmit this context through the traceparent header. Its format is:

<version>-<trace-id>-<parent-id>-<trace-flags>
Enter fullscreen mode Exit fullscreen mode

For example:

00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Enter fullscreen mode Exit fullscreen mode

Each field serves a specific purpose:

Field Description
version The Trace Context format version
trace-id The identifier shared by all spans in the distributed trace
parent-id The span ID of the calling operation
trace-flags Trace-related flags, including the sampling decision

When a service receives a request, it extracts the incoming context and uses it as the parent context for its new span. Before calling another service, it propagates the updated context. This process connects spans across the request path.

The following sequence shows how the API gateway, order service, and payment service propagate context.

A diagram displaying sequence shows how the API gateway, order service, and payment service propagate context.

The resulting trace represents the request across all three services. You can inspect it in a tracing backend such as Jaeger to follow the request and identify where an operation takes longer than expected.

What context propagation does not do

Context propagation carries context information, not telemetry data. It does not transfer log records or metric measurements between services. Each service must still generate and export its own telemetry.

It also does not guarantee that every signal will automatically appear together in an observability tool. Correlation requires the relevant trace context, resource information, instrumentation, and backend support.

Propagation also requires attention at service boundaries:

  • Asynchronous communication: Message queues and event-driven systems need to carry context through message headers or other supported metadata. Depending on the instrumentation, you might need additional configuration or manual propagation.
  • External services: Avoid propagating sensitive information to untrusted services. In particular, review what you include in baggage and define clear propagation boundaries.

Correlating logs, metrics, and traces

Two mechanisms are particularly useful for connecting logs and metrics to traces: trace identifiers in log records and exemplars in metrics.

Linking logs to traces

A log record can include the trace and span IDs active when the application emits it. These identifiers help an observability backend associate the log with the operation that generated it.

For example, the payment-service might emit the following log record:

{
  "timestamp": "2026-11-10T10:30:00Z",
  "severity_text": "ERROR",
  "body": "Payment provider request timed out",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7"
}
Enter fullscreen mode Exit fullscreen mode

This is an illustrative JSON representation. OpenTelemetry's log data model defines the corresponding fields as TraceId and SpanId. Exporters and logging integrations can map them to different field names and formats.

With these identifiers, you can open a trace, locate the relevant span, and inspect the associated log records. For example, a timeout message linked to a payment span can help you investigate why a checkout request failed.

This correlation requires a logging integration or another mechanism that captures the active trace context and adds the identifiers to log records. Not every logging framework does this automatically.

Linking metrics to traces with exemplars

Metrics help you identify patterns and changes in system behavior, such as increased request latency or error rates. However, an aggregated metric usually doesn't show which individual request contributed to a measurement.

Exemplars provide a way to investigate individual measurements. An exemplar associates a metric measurement with additional context, which can include a trace ID and span ID.

Consider an HTTP request duration histogram. The following is a simplified representation of the histogram and one of its exemplars:

Metric: http.server.request.duration

Aggregated measurements:
  Request count: 1,250
  Duration distribution: ...

Exemplar:
  Observed value: 2.4 seconds
  Trace ID: 4bf92f3577b34da6a3ce929d0e0e4736
  Span ID: 00f067aa0ba902b7
Enter fullscreen mode Exit fullscreen mode

The histogram helps you identify a latency problem across many requests. The exemplar provides a reference to a particular measurement and its trace context, so you can investigate the corresponding request.

Exemplars are optional. Their availability depends on instrumentation, SDK and exporter configuration, and backend support. The referenced trace must also be available in your tracing backend.

Avoid adding trace IDs or span IDs as metric labels. Because these identifiers can vary for every request, using them as labels creates high-cardinality time series and increases storage and query costs. Exemplars provide a more suitable way to link metric measurements to traces.

How the signals fit together

The following diagram distinguishes shared resource information from direct links between signals.

The following diagram distinguishes shared resource information from direct links between signals. <br>

Resource attributes help identify and group telemetry from the same entity. Logs can link directly to traces through trace and span identifiers, while metric exemplars provide references to trace context.

Together, these mechanisms help you move from an aggregated view of system health to the details of an individual request.

Common correlation gaps

Even with consistent instrumentation, implementation and configuration issues can make cross-signal correlation unreliable.

  • Context propagation stops at service boundaries: HTTP instrumentation might handle propagation automatically, but asynchronous messaging, custom protocols, and unsupported libraries can require additional configuration.
  • Logs don't contain trace context: A logging framework might not automatically include trace and span IDs. Verify that your logging integration captures the active context and exports the identifiers with log records.
  • Exemplars aren't available or preserved: Exemplar support depends on the SDK, exporter, and observability backend. Verify that your pipeline preserves exemplars and that your backend can use them to navigate to traces.
  • A missing trace is mistaken for an instrumentation failure: Sampling decisions, export failures, and retention policies can affect trace availability. Check these settings before concluding that instrumentation failed.
  • Separate backends don't support cross-signal navigation: Exporting metrics, logs, and traces to different destinations doesn't guarantee that you can navigate between them. Check whether your observability tools support the required correlation features and whether you have configured them correctly.

Validate telemetry correlation

Use this checklist when testing a new service or reviewing an existing telemetry pipeline:

  • [ ] Send a test request that crosses at least two services.
  • [ ] Verify that the resulting spans share a trace ID and have the expected parent-child relationships.
  • [ ] Confirm that the corresponding log records contain the expected trace and span identifiers.
  • [ ] Check whether metric exemplars link to traces that are available for inspection.
  • [ ] Test cross-signal navigation in your observability backend.
  • [ ] Repeat these checks across asynchronous and external service boundaries where applicable.

Dash0

Dash0 is an OpenTelemetry-native observability platform that brings metrics, logs, traces, and resource information into a unified interface. It also provides service monitoring, dashboards, alerting, and query capabilities.

Dash0 extends OpenTelemetry semantic conventions with its own dash0.* attributes, metrics, and events. It maintains a public registry that documents the meaning and usage of these fields.

Dash0 also supports semantic convention migrations during ingestion. These migrations help align incoming telemetry with a selected conventions version. For example, Dash0 can rename http.status_code to http.response.status_code or move service.name from span attributes to resource attributes.

Consider the checkout application from our earlier example. Users report slow checkout requests, and the operations team needs to identify the affected service.

Dash0 provides several tools to investigate the issue:

  1. Identify the problem: The Service Catalog helps identify checkout services with increased latency or error rates.
  2. Find the dependency: The Service Map shows service relationships and request flows, helping identify which downstream services need investigation.
  3. Inspect the request: The Trace Explorer lets you examine a representative trace and locate slow operations across the service path.
  4. Find supporting evidence: Inspect related logs for errors, such as payment provider timeouts, associated with the affected request.
  5. Verify the fix: After resolving the issue, compare latency and error metrics to confirm that the service has recovered.

The following diagram summarizes the workflow.

The following diagram displaying how to investigate data in Dash0.

A unified observability platform doesn't remove the need for correct instrumentation. Dash0 cannot reconstruct missing trace context or generate application-specific telemetry that your instrumentation doesn't capture. You still need to configure your OpenTelemetry SDKs and Collectors correctly.

You should also evaluate data residency, deployment requirements, telemetry volume, and compatibility with existing tools and workflows before adopting the platform.

Dash0 is worth considering when you want a managed, OpenTelemetry-native experience without operating multiple observability backends yourself. However, teams with specific infrastructure, compliance, or backend requirements should validate those needs before migrating.

For more information, see the Dash0 documentation.

Conclusion

Collecting metrics, logs, and traces gives you visibility into your applications. Connecting those signals gives you a more complete picture of system behavior.

OpenTelemetry provides several building blocks for doing this:

  • Semantic conventions establish common names and meanings for telemetry.
  • Resource attributes identify the entities producing telemetry.
  • Context propagation connects operations across service boundaries.
  • Log trace context and metric exemplars provide ways to navigate between signals.

These mechanisms don't remove the need for correct instrumentation or backend configuration. They provide the shared structure and context that make correlation possible.

When telemetry follows consistent conventions and carries the right context, you spend less time figuring out what each data point represents and more time understanding what happened.

Top comments (0)