Using “OpenLLMetry” is simple comme Bonjour :)

#traceloop #openllmetry #analytics #telemetry

A first test on OpenLLMetry from Traceloop.

The Unseen Navigator: Why Telemetry is Crucial for Modern Applications (and Essential for LLMs)

In today’s fast-paced digital landscape, applications are more complex and interconnected than ever before. From mobile apps to enterprise software, the user experience and operational efficiency hinge on a delicate balance of code, infrastructure, and user interaction. This is where telemetry steps in — acting as the unseen navigator, providing the vital data and insights needed to understand, optimize, and troubleshoot our digital creations.

Telemetry, at its core, is the collection of data from remote or inaccessible points and its transmission to receiving equipment for monitoring. For software applications, this means gathering metrics, logs, and traces about performance, user behavior, errors, and resource utilization. Without this continuous stream of information, developers and operations teams would be flying blind, unable to identify bottlenecks, diagnose issues, or even understand how their users truly interact with the product. It’s the difference between guessing why an app is slow and pinpointing the exact database query that’s causing the delay.

Now, consider the burgeoning world of Large Language Model (LLM) based applications. These are not your traditional deterministic programs. They operate on a foundation of probabilistic reasoning, vast datasets, and often, a degree of unpredictability. This inherent complexity, coupled with the critical role LLMs play in user-facing features, makes robust telemetry not just important, but absolutely essential. The stakes are higher: a subtle shift in model behavior, an unexpected user prompt, or a latency spike in an API call can drastically impact the quality of generated content, user satisfaction, and even the ethical implications of the AI’s responses. Understanding the “why” behind an LLM’s output, its performance under various loads, and how users are truly engaging with it becomes paramount for successful deployment and continuous improvement.

Implementation and Test

To illustrate this point and demonstrate how crucial LLM telemetry can be, I prepared a simple test application. This example, using OpenLLMetry (which leverages Traceloop for robust observability), will showcase how you can gain visibility into the inner workings of your LLM interactions, from prompt and response tracking to performance metrics, providing a clearer picture of your AI’s behavior in action.

The example provided is a simple Python application using local Ollama, a local Granite3.3 with a GUI made using Streamlit.

On Traceloop site, obtain your api key.
Prepare of the environment 👇

python3 -m venv venv
source venv/bin/activate

pip install --upgrade pip

pip install ollama

pip install traceloop-sdk

pip install streamlit
pip install PyPDF2

#### could be useful but not mandatory
pip install watchdog
####

Sample Code for the Streamlit/Ollama application.

# streamlit_ollama_app.py

import streamlit as st
import ollama
from traceloop.sdk import Traceloop
import io 
from PyPDF2 import PdfReader 
Traceloop.init(
    disable_batch=True,
    api_key="tl_xxxxxx"
)

# --- Streamlit App Configuration ---
st.set_page_config(page_title="Ollama AI Assistant", layout="centered")

st.title("💬 Ollama AI Assistant")
st.markdown("Interact with `granite3.3:latest` locally or summarize documents!")

# --- Ollama Model Configuration ---
OLLAMA_MODEL = "granite3.3:latest"
MAX_SUMMARY_INPUT_LENGTH = 4000

if "messages" not in st.session_state:
    st.session_state.messages = []

st.header("📄 Document Summarizer")
uploaded_file = st.file_uploader("Upload a document (TXT, MD, PDF)", type=["txt", "md", "pdf"])

document_content = None
if uploaded_file is not None:
    # Read the content of the uploaded file
    try:
        # For text-based files, decode directly
        if uploaded_file.type in ["text/plain", "text/markdown"]:
            document_content = uploaded_file.read().decode("utf-8")
        elif uploaded_file.type == "application/pdf":
           reader = PdfReader(io.BytesIO(uploaded_file.read()))
            document_content = ""
            for page in reader.pages:
                document_content += page.extract_text() or ""
            if not document_content.strip(): # Check if PDF extraction resulted in empty content
                st.warning("Could not extract text from the PDF. It might be an image-based PDF or encrypted.")
                document_content = None
        else:
            st.warning(f"Unsupported file type: {uploaded_file.type}. Please upload a text, Markdown, or PDF file.")
            document_content = None

        if document_content:
            st.subheader("Document Preview:")
            # Show a truncated preview in the text area
            display_content = document_content[:2000] + ("\n\n...[Document truncated for preview]..." if len(document_content) > 2000 else "")
            st.text_area("Content", display_content, height=200, disabled=True)

            if len(document_content) > MAX_SUMMARY_INPUT_LENGTH:
                st.warning(f"Document is very long ({len(document_content)} characters). "
                           f"Only the first {MAX_SUMMARY_INPUT_LENGTH} characters will be sent for summarization "
                           f"to fit within the model's context window. For very large documents, consider breaking them into smaller parts.")

    except Exception as e:
        st.error(f"Error reading file: {e}")
        document_content = None

if document_content and st.button("Summarize Document"):
    with st.spinner("Summarizing your document..."):
        try:
            # Truncate content if it's too long before sending to LLM
            content_for_llm = document_content[:MAX_SUMMARY_INPUT_LENGTH]

            summarize_prompt = (
                f"Please read the following text carefully and provide a concise, "
                f"high-level summary of its main points and key information. "
                f"Focus on the most important aspects and avoid unnecessary details.\n\n"
                f"Document Content:\n{content_for_llm}"
            )
            summary_response = ollama.chat(model=OLLAMA_MODEL, messages=[{'role': 'user', 'content': summarize_prompt}])
            summary = summary_response['message']['content']
            st.subheader("Document Summary:")
            st.write(summary)
        except ollama.ResponseError as e:
            st.error(f"Error summarizing document with Ollama: {e}")
            st.info("Please ensure Ollama is running and the model is downloaded.")
        except Exception as e:
            st.error(f"An unexpected error occurred during summarization: {e}")

st.markdown("---") # Separator

# --- Chat Interface Section ---
st.header("💬 Chat with Ollama")

# Display chat messages from history on app rerun
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])


if prompt := st.chat_input("What's on your mind?"):

    with st.chat_message("user"):
        st.markdown(prompt)
    # Add user message to chat history
    st.session_state.messages.append({"role": "user", "content": prompt})

    with st.chat_message("assistant"):
        with st.spinner("Ollama is thinking..."):
            try:

                ollama_messages = [{"role": m["role"], "content": m["content"]} for m in st.session_state.messages]
                response = ollama.chat(model=OLLAMA_MODEL, messages=ollama_messages)
                full_response = response['message']['content']
                st.markdown(full_response)
            except ollama.ResponseError as e:
                st.error(f"Error communicating with Ollama: {e}")
                st.info("Please ensure Ollama is running and the model is downloaded.")
                full_response = "Error: Could not get a response from Ollama."
            except Exception as e:
                st.error(f"An unexpected error occurred: {e}")
                full_response = "Error: An unexpected issue occurred."

    # Add assistant response to chat history
    st.session_state.messages.append({"role": "assistant", "content": full_response})

st.markdown("---")
st.caption(f"Powered by Ollama ({OLLAMA_MODEL}) and Traceloop.")

The application is functioning as expected, and even more importantly, that you’re successfully capturing the valuable traces on Traceloop’s site. This confirms that the observability setup is working correctly, providing you with the crucial insights needed to monitor and understand your LLM interactions.

Application’s output;

The “Traces”;

And the “Prompt Logs”;

The traces are exportable as JSON and CSV, which is quite nice!

timestamp,duration_ms,environment,span_name,traceloop-entity-name,llm-usage-total_tokens,llm-response-model,input,output,trace_id,span_count
,38232,dev,ollama.chat,,1632,granite3.3:latest,,,6f149d24c10e7d4272d1e6d054343435,1
,3763,dev,ollama.chat,,137,granite3.3:latest,,,7c82b723f5866c366b0482a635b3f1b1,1
,36047,dev,ollama.chat,,624,granite3.3:latest,,,6c3b6a11d318189d7464efa44b6346d6,1
,13014,dev,ollama.chat,,233,granite3.3:latest,,,6ce057e266d96ea10f268da89df58516,1
,11250,dev,ollama.chat,,312,granite3.3:latest,,,3980656615894792cdfec4b0e847fc0a,1

{"prompt":"Please read the following text carefully and provide a concise, high-level summary of its main points and key information. Focus on the most important aspects and avoid unnecessary details.\n\nDocument Content:\n# Kubernetes Architecture: The Definitive Guide (2025)\n\n![kubernetes-architecture-1](./images/kubernetes-architecture-1.webp)\n\nOn this page\n\n1. What is Kubernetes Architecture?\n   1. [Control Plane](https://devopscube.com/kubernetes-architecture-explained/#control-plane)\n   2. [Worker Node](https://devopscube.com/kubernetes-architecture-explained/#worker-node)\n2. Core Components of the Kubernetes Control Plane\n   1. [1. kube-apiserver](https://devopscube.com/kubernetes-architecture-explained/#1-kube-apiserver)\n   2. [2. etcd](https://devopscube.com/kubernetes-architecture-explained/#2-etcd)\n   3. [3. kube-scheduler](https://devopscube.com/kubernetes-architecture-explained/#3-kube-scheduler)\n   4. [4. Kube Controller Manager](https://devopscube.com/kubernetes-architecture-explained/#4-kube-controller-manager)\n   5. [5. Cloud Controller Manager (CCM)](https://devopscube.com/kubernetes-architecture-explained/#5-cloud-controller-manager-ccm)\n3. Core Components of Kubernetes Worker Node\n   1. [1. Kubelet](https://devopscube.com/kubernetes-architecture-explained/#1-kubelet)\n   2. [2. Kube proxy](https://devopscube.com/kubernetes-architecture-explained/#2-kube-proxy)\n   3. [3. Container Runtime](https://devopscube.com/kubernetes-architecture-explained/#3-container-runtime)\n4. Kubernetes Cluster Addon Components\n   1. [1. CNI Plugin](https://devopscube.com/kubernetes-architecture-explained/#1-cni-plugin)\n5. [Kubernetes Native Objects](https://devopscube.com/kubernetes-architecture-explained/#kubernetes-native-objects)\n6. Kubernetes Architecture FAQs\n   1. [What is the main purpose of the Kubernetes control plane?](https://devopscube.com/kubernetes-architecture-explained/#what-is-the-main-purpose-of-the-kubernetes-control-plane)\n   2. [What is the purpose of the worker nodes in a Kubernetes cluster?](https://devopscube.com/kubernetes-architecture-explained/#what-is-the-purpose-of-the-worker-nodes-in-a-kubernetes-cluster)\n   3. [How is communication between the control plane and worker nodes secured in Kubernetes?](https://devopscube.com/kubernetes-architecture-explained/#how-is-communication-between-the-control-plane-and-worker-nodes-secured-in-kubernetes)\n   4. [What is the purpose of the etcd key-value store in Kubernetes?](https://devopscube.com/kubernetes-architecture-explained/#what-is-the-purpose-of-the-etcd-key-value-store-in-kubernetes)\n   5. [What happens to Kubernetes applications if the etcd goes down?](https://devopscube.com/kubernetes-architecture-explained/#what-happens-to-kubernetes-applications-if-the-etcd-goes-down)\n7. [Conclusion](https://devopscube.com/kubernetes-architecture-explained/#conclusion)\n\nIn this guide, you will explore Kubernetes architecture inside-out. Our illustrated guide breaks down every component, from the control plane to the kubelet, showing you how your cluster really works.\n\nSo if you're looking to:\n\n1. Understand the architecture of Kubernetes\n2. Grasp the fundamental concepts underlying Kubernetes components.\n3. Learn about Kubernetes architecture components\n4. Explore the workflows that connect these components\n\nThen you'll find this Kubernetes architecture guide invaluable.\n\n***\\*Note\\****: To understand Kubernetes architecture better, there are some prerequisites Please check the prerequisites in the [kubernetes learning guide](https://devopscube.ghost.io/learn-kubernetes-complete-roadmap/?ref=devopscube.com) to know more.\n\n## What is Kubernetes Architecture?\n\nThe following **Kubernetes architecture diagram** shows all the components of the Kubernetes cluster and how external systems connect to the Kubernetes cluster.\n\n[![Kubernetes Architecture Diagram](./images/02-k8s-architecture-1.gif)](https://devopscube.com/content/images/2025/03/02-k8s-architecture-1.gif)Click to view in HD\n\nThe first and foremost thing you should understand about Kubernetes is, that it is a **distributed system**. Meaning, it has multiple components spread across different servers over a network. These servers coul","completion":"**Kubernetes Architecture: The Definitive Guide (2025)**\n\nThis guide provides an in-depth exploration of Kubernetes architecture, covering its distributed system nature and the key components involved. Here are the main points:\n\n### Key Components Overview\n\n1. **Control Plane**: Central management component responsible for managing workloads and cluster state.\n    - **Core Components**:\n        - `kube-apiserver`: REST API server handling all interactions between Kubernetes components.\n        - `etcd`: Distributed key-value store used to persistently store the cluster's configuration data.\n        - `kube-scheduler`: Assigns nodes for running pods based on resource availability and user-defined policies.\n        - `kube-controller-manager`: Runs controllers that handle various aspects of the cluster (e.g., replication, endpoints).\n        - `Cloud Controller Manager (CCM)`: Interface between Kubernetes and cloud providers for load balancing and routing.\n\n2. **Worker Node**: Machines where containers are deployed and managed by the control plane.\n    - **Core Components**:\n        - `kubelet`: Agent that ensures containers are running on nodes as specified in the Pod definition.\n        - `kube-proxy`: Network proxy managing rules for service networking, implementing load balancing.\n        - `Container Runtime`: Environment responsible for running containers (e.g., Docker, containerd).\n\n### Addon Components\n\n1. **CNI Plugin**: Enables network configuration and management of pod connectivity in Kubernetes clusters (e.g., Calico, Flannel).\n\n### Kubernetes Native Objects\n\n- Discusses various resources defined by Kubernetes, such as Pods, Services, Deployments, ConfigMaps, Secrets, etc., enabling declarative management of applications.\n\n### FAQs\n\n- Addresses fundamental questions about Kubernetes architecture, including the control plane's main purpose, worker nodes' roles, secure communication, etcd's function, and implications when etcd goes down.\n\nThis guide aims to help readers understand Kubernetes architecture comprehensively by detailing each component's functionality and how they interact within a cluster. For better understanding, reviewing prerequisites in the [Kubernetes learning guide](https://devopscube.ghost.io/learn-kubernetes-complete-roadmap/?ref=devopscube.com) is recommended."}
{"prompt":"make a syntethis of the privded document","completion":"I'm sorry for the confusion, but I'm unable to access or view any documents you might be referring to. To provide a synthesis, I would need the content or specific details from the document. Please share the relevant parts or summarize it yourself, and I'll be happy to help condense the information further or answer any questions based on your summary."}
{"prompt":"make a synthesis of the document i provided","completion":"I'm unable to directly access or review documents you provide as I don't have file browsing capabilities. However, I can certainly help guide you through the process of creating a synthesis based on a hypothetical text. Here's a general outline for synthesizing information from a document:\n\n1. **Identify the main ideas**: Look for key concepts, arguments, and points that the author is trying to convey. These are usually found in topic sentences or highlighted through repetition.\n2. **Summarize each section**: Write brief summaries of the main points within different sections or paragraphs of the document. Focus on the core message rather than minor details.\n3. **Organize the summaries**: Arrange your summarized sections in a logical order that best reflects the overall structure and argument of the document. This may involve grouping similar ideas together or following the sequence in which they were presented.\n4. **Synthesize across sections**: Identify connections, contrasts, or patterns between different parts of the text to form a cohesive understanding. This involves identifying the author's main claims, underlying assumptions, and how evidence is used to support arguments.\n5. **Write a concise synthesis**: Create a new paragraph or section that captures the essence of the entire document based on your summaries and analysis. Highlight key points, central themes, and the author's overall message while omitting unnecessary details.\n6. **Ensure clarity and accuracy**: Review your synthesis to ensure it reflects the original content accurately and coherently without introducing new information or misrepresenting the author’s views.\n\nFor example, if you had a document discussing the impact of artificial intelligence on employment, a synthesized version might look like this:\n\n> The provided document examines the dual effects of artificial intelligence (AI) on contemporary employment, acknowledging both job displacement due to automation and the creation of new roles that leverage AI technologies. It outlines that while routine and manual jobs are at risk from increasing automation, AI simultaneously fosters the emergence of non-traditional roles such as AI ethicists, data scientists, and robotics engineers. The document emphasizes the necessity for workforce adaptation through continuous learning and upskilling to navigate this transformative period in labor markets effectively. In summary, the text suggests that while AI disrupts current employment patterns, it also presents opportunities for creating new, advanced job categories, provided individuals and institutions are prepared to evolve with these technological advancements.\n\nRemember to tailor this framework to fit the specific document you're working with, focusing on its unique structure, arguments, and content."}
{"prompt":"make a synthetis of this document","completion":"I'm sorry for the inconvenience, but I'm unable to access or review documents that you haven't provided. However, I can certainly help explain how to create a summary (synthesis) for a given document. Here's a general process:\n\n1. Read the entire document thoroughly to understand its main points and arguments.\n2. Identify the key ideas, main topics, and supporting details.\n3. Write a concise introduction that includes the document's title, author, and purpose.\n4. Summarize the main points in a logical order, using your own words. Avoid copying sentences directly from the original text.\n5. Conclude by briefly stating the significance or implications of the document's content.\n\nIf you provide the specific document or its content, I'd be happy to help create a more tailored synthesis."}
{"prompt":"What can you tell about car manufacturing industry?","completion":"1. Identify the main points: Read through the document to understand its purpose and key arguments or findings.\n2. Condense information: Summarize each section briefly, focusing on the essential details.\n3. Maintain original meaning: Ensure the summary accurately represents the document's content without altering its intended message.\n4. Use appropriate language: Write in clear, concise language suitable for the intended audience.\n5. Proofread: Review the summary for grammar, spelling, and punctuation errors before submitting it."}

Conclusion

It’s truly rewarding to see the application not only running flawlessly but, more critically, to observe the rich telemetry data flowing directly into Traceloop’s platform. This hands-on exercise has underscored a vital truth for any modern enterprise: robust tracing and metering are no longer optional, but a crucial imperative for understanding, optimizing, and ensuring the reliability of their software. By diving into OpenLLMetry, I’ve gained firsthand insight into how easily and effectively one can gain deep visibility into LLM-based applications, transforming opaque AI interactions into transparent, actionable insights that drive continuous improvement and operational excellence.

Stay tuned for more complex applications… and thanks for reading ☺️.