DEV Community

Cover image for I Built a Self-Hosted LLM Observability Tool for AI Applications (Logmera)
Thilak Kumar
Thilak Kumar

Posted on

I Built a Self-Hosted LLM Observability Tool for AI Applications (Logmera)

#python #ai #opensource #devtools

When building AI applications, one problem appears very quickly:

You lose visibility into what your AI system is doing.

Questions start appearing:

  • What prompts were sent to the model?
  • What responses came back?
  • How long did the request take?
  • Which model handled the request?
  • Why did a request fail?

Most developers initially log things to the console, but that quickly becomes messy in production.

So I built a small tool called Logmera.

What is Logmera?

Logmera is a self-hosted observability tool for AI / LLM applications.

Instead of printing logs to the console, Logmera stores:

  • prompts
  • responses
  • model name
  • latency
  • request status

in a PostgreSQL database, and shows them in a simple web dashboard.

The idea is simple:

Your AI app sends logs → Logmera stores them → you inspect them in the dashboard.

Why I Built It

Many LLM observability tools require sending prompts and responses to external cloud services.

For some teams that is fine, but in other cases it raises concerns about:

  • privacy
  • compliance
  • data ownership

Logmera takes a different approach:

Everything runs on your own infrastructure.

Your logs stay in your PostgreSQL database.

How Logmera Works 

Your AI Application
        │
        ▼
Logmera Python SDK
        │
        ▼
Logmera Server (FastAPI)
        │
        ▼
PostgreSQL Database
        │
        ▼
Dashboard
Enter fullscreen mode Exit fullscreen mode

Your application logs AI requests using a small Python SDK, and Logmera stores and visualizes them.

Quick Start

You can get Logmera running in about 2 minutes.

1. Install 

pip install logmera
Enter fullscreen mode Exit fullscreen mode

2. Start the server

Logmera needs a PostgreSQL database.

Start the server like this:

logmera --db-url "postgresql://username:password@localhost:5432/database"
Enter fullscreen mode Exit fullscreen mode

The server starts at:

http://127.0.0.1:8000
Enter fullscreen mode Exit fullscreen mode

3. Log an AI request

Add a single line of logging to your AI code.

import logmera

logmera.log(
    project_id="chatbot",
    prompt="Hello",
    response="Hi there",
    model="gpt-4o",
    latency_ms=120,
    status="success"
)
Enter fullscreen mode Exit fullscreen mode

Now the request appears in the dashboard.

Dashboard

Logmera includes a simple dashboard where you can:

  • browse logs
  • search prompts
  • filter by project
  • filter by model
  • track latency
  • inspect responses

This makes debugging AI systems much easier.

API Support

Logmera also exposes a REST API so logs can be sent from any language.

Example:

curl -X POST http://127.0.0.1:8000/logs \
-H "Content-Type: application/json" \
-d '{
  "project_id":"demo",
  "prompt":"Hello",
  "response":"Hi",
  "model":"gpt-4o",
  "latency_ms":95,
  "status":"success"
}'
Enter fullscreen mode Exit fullscreen mode

Who Is This Useful For?

Logmera can help if you're building:

  • AI SaaS applications
  • chatbots
  • RAG systems
  • AI agents
  • automation tools powered by LLMs

It provides simple visibility into what your AI system is doing.

Links

PyPI
https://pypi.org/project/logmera/

GitHub
https://github.com/ThilakKumar-A/Logmera/

If you're building AI applications, I would love to hear feedback.

Top comments (0)