DEV Community

Glenn Allogho
Glenn Allogho

Posted on Originally published at pub.towardsai.net

Your LLM tracing tool isn't where the lock-in is

#opentelemetry #observability #ai #llm

Langfuse, Phoenix and LangSmith all accept OpenTelemetry spans over OTLP. If your app emits plain spans, the backend is a config choice:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:6006 python trace_demo.py
Enter fullscreen mode Exit fullscreen mode

So tracing isn't the decision. Scores, datasets and prompts are, because no exporter setting moves them. That leaves four constraints worth picking on:

  • License. Langfuse is MIT outside ee/. Phoenix's server is Elastic License 2.0, so no hosting it for clients. LangSmith is closed, self-host is Enterprise only.
  • Ops. Self-hosted Langfuse is six containers. Phoenix is one process on SQLite.
  • Billing shape. Langfuse Cloud bills traces plus observations plus scores. Cheap at 4 spans per trace, more expensive than LangSmith at 40.
  • Eval loop. Wherever your evals run is where you'll stay.

The full comparison, a runnable trace_demo.py, and the pricing math at three traffic shapes: Langfuse vs Phoenix vs LangSmith: the decision is not tracing

Top comments (0)