DEV Community

Cover image for Streamline your OpenAI Monitoring Experience with GreptimeAI
Greptime
Greptime

Posted on

Streamline your OpenAI Monitoring Experience with GreptimeAI

With the rapid advancement of artificial intelligence technology, OpenAI has emerged as one of the leaders in the field. It excels in various language processing tasks, including machine translation, text classification, and text generation.

However, the critical role of continuous monitoring of API calls while using OpenAI should not be underestimated. This practice is crucial not only for identifying performance bottlenecks and analyzing usage patterns but also for swiftly detecting and addressing any issues that arise with the API.

GreptimeAI

GreptimeAI offers a tailor-made observability solution specifically designed for monitoring and managing large language model (LLM) applications. This solution provides comprehensive insights into the cost, performance, traffic, and security aspects of OpenAI usage. For more details about GreptimeAI, please refer to this article. Notably, GreptimeAI is built upon the open-source time-series database, GreptimeDB.

OpenAI Modules being Monitored

  • chat
  • completion
  • audio
  • images

Scenarios Supported

  • async
  • stream
  • with_raw_response
  • retry
  • error

User Guide

Installation

pip install --upgrade greptimeai
Enter fullscreen mode Exit fullscreen mode

Registration

To get started, create a service by registering greptimeai, and get:

  • host
  • database
  • token

Setting up

export GREPTIMEAI_HOST='xxx'
export GREPTIMEAI_DATABASE='xxx'
export GREPTIMEAI_TOKEN='xxx'
Enter fullscreen mode Exit fullscreen mode

Example

Here is a simple example to illustrate how to call OpenAI chat completion with GreptimeAI tracking enabled.

from greptimeai import openai_patcher
from openai import OpenAI

client = OpenAI()
openai_patcher.setup(client=client)

completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "How do I output all files in a directory using Python?",
        }
    ],
    model="gpt-4",
    user="<user_id_from_your_application>",
    stream=True,
)
Enter fullscreen mode Exit fullscreen mode

How does it look like in GreptimeAI

Dashboard overview:

Image description

The following graph shows the trace detail with multiple spans.

Image description

API Trace View

How I Cut 22.3 Seconds Off an API Call with Sentry 🕒

Struggling with slow API calls? Dan Mindru walks through how he used Sentry's new Trace View feature to shave off 22.3 seconds from an API call.

Get a practical walkthrough of how to identify bottlenecks, split tasks into multiple parallel tasks, identify slow AI model calls, and more.

Read more →

Top comments (0)

The Most Contextual AI Development Assistant

Pieces.app image

Our centralized storage agent works on-device, unifying various developer tools to proactively capture and enrich useful materials, streamline collaboration, and solve complex problems through a contextual understanding of your unique workflow.

👥 Ideal for solo developers, teams, and cross-company projects

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay