Monitor LLM Costs with Prometheus & Grafana (Without a Proxy)

PLATAFORMA: Dev.to

If you're running any observability stack, you've probably got Prometheus and Grafana at the core. You've got your CPU usage, your memory, your network I/O... but what about your single biggest, most volatile line item? Your LLM bill.

Getting that data into your existing dashboards is a pain. Most solutions want you to route all your traffic through their proxy, adding a point of failure and latency you don't need.

It doesn't have to be that complicated.

I'm a big believer in async, out-of-band monitoring. Your app should fire and forget logging events, not wait for a monitoring service to respond. This is why when I built LLMeter, I made sure it could plug into the tools you already use.

Here's how you can pipe your LLM costs into Prometheus without a proxy.

LLMeter exposes a /metrics endpoint that Prometheus can scrape, just like it does for your other services. All you need to do is add a new job to your prometheus.yml:

scrape_configs:
  - job_name: 'llmeter'
    scrape_interval: 15s
    static_configs:
      - targets: ['your-llmeter-instance.local:3000']

That's it. Now you can build Grafana dashboards that show your LLM costs right next to your other metrics. You can see how a spike in traffic to a certain endpoint correlates with a spike in your Anthropic bill. You can finally see the real cost of that new AI feature you just shipped.

This approach gives you:

• Zero-latency monitoring: Your API calls go straight to the provider, not through a third-party proxy.
• Real-time visibility: See your costs as they happen, not at the end of the month.
• Unified dashboards: No more context-switching between your observability stack and your LLM provider's billing page.

Stop flying blind. Your LLM bill is part of your infrastructure cost. It's time to start treating it that way.