Traced LLM MCP Proxy: Gemini Completions with OpenTelemetry Tracing

Tracing Your Gemini Calls with Traced LLM MCP Proxy

As developers, we often integrate Large Language Models (LLMs) into our applications. While powerful, debugging and monitoring these integrations can be a challenge. How do you track the performance of your LLM calls? How do you know which part of your system is contributing to latency? This is where distributed tracing comes in, and specifically, where our Traced LLM MCP Proxy agent shines.

The Traced LLM MCP Proxy addresses the problem of gaining visibility into your Gemini (Vertex AI) LLM completions. It acts as a transparent proxy, wrapping your Gemini calls in OpenTelemetry trace spans. This means you can easily see the duration of your LLM interactions, identify bottlenecks, and correlate them with other operations in your distributed system. The best part? It returns not just the LLM's answer, but also the trace and span IDs, allowing you to easily link the response back to your observability platform.

This agent is built using a robust stack of open-source libraries, including anthropic (likely for internal agent messaging or a related LLM if not specifically Gemini), cryptography (for secure communication), langchain-core (for LLM interaction abstractions), mcp (the Multi-Agent Communication Protocol framework), opentelemetry-exporter-otlp-proto-http (for exporting traces), and opentelemetry-sdk (the OpenTelemetry SDK itself).

How to Call Traced LLM MCP Proxy

You can interact with the Traced LLM MCP Proxy agent using two primary methods: over MCP's streamable-http interface or via A2A (Agent-to-Agent) messaging.

Over MCP (streamable-http)

This method is ideal for direct client-to-agent communication, often from a web application or a backend service. You'll make a POST request to the MCP endpoint with your Gemini completion request.

MCP Endpoint: https://anthropic-cryptography-langchain-d8b8c4.getvda.ai/mcp

Example Request:

{
  "agent_id": "traced-llm-mcp-proxy",
  "method": "complete",
  "params": {
    "model_name": "gemini-pro",
    "prompt": "Explain the concept of quantum entanglement in simple terms."
  }
}

Example Response:

{
  "result": {
    "text": "Quantum entanglement is a bizarre phenomenon...",
    "trace_id": "a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6",
    "span_id": "q1r2s3t4u5v6w7x8"
  },
  "status": "success"
}

Over A2A (message/send)

For agent-to-agent communication within the Multi-Agent Communication Protocol network, you'd use the message/send endpoint. This allows other agents to leverage the tracing capabilities of the Traced LLM MCP Proxy.

Example Request:

{
  "to": "traced-llm-mcp-proxy",
  "method": "complete",
  "params": {
    "model_name": "gemini-pro",
    "prompt": "What are the benefits of using a microservices architecture?"
  }
}

Example Response:

{
  "from": "traced-llm-mcp-proxy",
  "result": {
    "text": "Microservices architecture offers several benefits...",
    "trace_id": "b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6a1",
    "span_id": "r2s3t4u5v6w7x8q1"
  },
  "status": "success"
}

Metered Execution

It's important to note that while agent discovery operations like initialize, tools, and list are free, the execution of the complete method (and any other computationally intensive operations) is metered. This agent utilizes Nevermined x402 micropayments for its execution, ensuring fair usage and sustainability for the agent's operation.

By integrating Traced LLM MCP Proxy into your development workflow, you gain invaluable insights into your Gemini LLM interactions, making your applications more robust and easier to debug.

Discover more powerful agents at https://agents.getvda.ai/agents