Y.Matsuda

Posted on Jun 29

otel-tui: A TUI Tool for Viewing OpenTelemetry Traces

#observability #opentelemetry #webdev

Introduction

The OpenTelemetry ecosystem has grown significantly busier compared to a while ago. I've been contributing and writing blog posts about it for about two years now, and I'm hoping to participate in some synchronous events in the future.

Now, when it comes to instrumenting applications or writing libraries for instrumentation, how do you usually verify the local operation?

From my experience, many people run Jaeger or Grafana locally to check signals. Today, I’d like to introduce a new method: my custom tool "otel-tui," which allows you to view OpenTelemetry in your terminal!

https://github.com/ymtdzzz/otel-tui

Motivation

The Need for a "Just Right" Tool for Local Development

Based on my experience, here are some use cases for verifying signals locally:

Ensuring traces are linked as intended
- For example, verifying implementation at the level of context
Checking that attributes are set as expected
- Attributes specified by the vendor may not display correctly if missing
- Custom attributes may be specified

Currently, there are several ways to verify signals (mainly traces) locally:

Run Jaeger (all-in-one with UI) or Grafana Tempo containers locally to send and view signals
Use an exporter to output to standard output or files

However, these methods have limitations:

Running Jaeger or Grafana Tempo containers locally is resource-intensive
Production-grade features are often too much for local use (e.g., service maps)
It’s challenging to view outputs in real-time

Using an exporter to output to standard output or files also has limitations:

Low readability (e.g., grepping for Trace ID or parent Span ID to check trace linkage)

Given these challenges, I wanted a tool that could offer the best of both worlds—one that wasn’t too heavy for local use and provided readable outputs.

There're Similar Tools but...

There’s also "otel-desktop-viewer," which focuses on viewing OpenTelemetry signals locally and is very user-friendly.

However, with a significant volume of data over a long period, it can become sluggish in terms of query and screen rendering (depending on machine specs). This inspired me to create my own tool that could view data in real-time without opening a browser and maintain stable performance.

Features

Here's what the screen looks like:

Key features include:

Traces
- List view of service spans and search by service name
- Detailed attribute information
- Trace graph display (waterfall diagram)
Logs
- List view and search
- Jump from logs to corresponding traces
Performance
- Lightweight and real-time screen updates (asynchronous signal reception and rendering)
- No memory leaks even with a steady flow of data over a long period (data rotation)

TODOs

There are still many features missing:

Flexible settings
- Screen update frequency
- Buffer size
Metric collection and display
Stability
- Occasional crashes after prolonged use
Consistent keybindings
UI improvements
- Navigation (e.g., back and forth between traces and log details)
- Text wrapping, waterfall diagram
- Debug logs

How to Use

To demonstrate usage, I’ll use signals from the "opentelemetry-demo."

Installing otel-tui

You can install it via Homebrew:

$ brew install ymtdzzz/tap/otel-tui

If Homebrew isn’t available, download the binary from the release page or build from source.

Verify installation with:

$ otel-tui -v
otel-tui version 0.1.2

$ otel-tui -h
Usage:
  otel-tui [flags]

Flags:
      --grpc int      The port number on which we listen for OTLP grpc payloads (default 4317)
  -h, --help          help for otel-tui
      --host string   The host where we expose all endpoints (OTLP receivers and browser) (default "0.0.0.0")
      --http int      The port number on which we listen for OTLP http payloads (default 4318)
  -v, --version       version for otel-tui

Changing the Target of the Instrumented Application

Clone the "opentelemetry-demo" repository and configure it.

Modify Collector Target

The opentelemetry-demo consists of multiple microservices, all signals are aggregated in the OpenTelemetry Collector. Modify the collector's exporter target to point to otel-tui.

# src/otelcollector/otelcol-config-extras.yml
exporters:
  otlp:
    endpoint: host.docker.internal:4317

service:
  pipelines:
    traces:
      exporters: [spanmetrics, otlp]
    logs:
      exporters: [otlp]

Allow Container to Host Port Communication (if not using Docker Desktop)

Set up host.docker.internal to be resolvable from within the container.

# docker-compose.minimal.yml
# OpenTelemetry Collector
otelcol:
  image: ${COLLECTOR_CONTRIB_IMAGE}
  container_name: otel-col
  # ...
  extra_hosts:
    - "host.docker.internal:host-gateway"

Starting otel-tui and the Instrumented Application

Start them in separate tabs.

# From any directory where otel-tui is in the PATH
$ otel-tui

# From the root of the opentelemetry-demo repository
$ make start-minimal

Exploring Data

After a short while, signals should start flowing into otel-tui.

You can switch between traces and logs using the Tab key. Navigate through panes using keys listed in the titles (e.g., Details(d)).

To filter the spans, use the search box (/ key) and search for something like "product."

Spans related to services with the prefix "product" are filtered. Details of the currently focused span are displayed in the Details pane. You can collapse the tree in the Details pane by pressing Enter.

Select a span in the Traces pane by pressing Enter to view the trace details.

Navigate the Trace Timeline using arrow keys. Details of the focused span appear in the Details pane. Related logs appear in the Logs pane.

Press Esc to return to the trace list, and Tab to switch to the logs view.

Search for a term and inspect the details. If a log has a linked trace (indicated by a link emoji 🔗), press Enter to jump to the trace details.

This tool helps verify if the instrumented application sends the intended information and if it’s correctly linked.

Conclusion

While there are still many rough edges and missing features, I find it quite user-friendly and enjoyable to use.

I’d appreciate any feedback via https://github.com/ymtdzzz/otel-tui/issues or https://twitter.com/ymtdzzz!

Top comments (6)

Jonathan VUILLEMIN • Jun 29

I love it 😍 Thx for sharing!

Will test it with Yokai demo apps. For the metrics it'll be otel or prom metrics that you have in mind?

Y.Matsuda • Jun 30

Thx, I hope you try it out 💖

I plan to support otel metrics. However, as this tool is just implemented as a custom otel exporter, it should be possible to use it even if you are using prom metrics with components such as a Prometheus receiver 👍

Matin Mollapur • Jun 29

wow, this looks super cool! i've been struggling with viewing traces locally, so this tool seems like a game-changer. can't wait to try it out on my projects. thanks for sharing!