Is Your OpenAI Bill Giving You Nightmares? I Built a Tool to Help

#programming #ai #npm #openai

Let's be honest: playing with large language models is amazing, but seeing that OpenAI API bill at the end of the month can be... painful. 😅

I've been working with the GPT-4 and GPT-3.5 APIs a lot, and I noticed how quickly the costs can spiral out of control. A simple task routed to GPT-4 by mistake, an inefficient prompt, or running the same query over and over—it all adds up.

I kept thinking there had to be a smarter, more automated way to manage this without rewriting all my code.

That's why I built CostLens, a simple SDK I'm hoping can help other developers who are facing the same problem.

What is CostLens?

At its core, CostLens is a drop-in SDK that automatically helps you cut your AI costs. The goal is to make it a "set it and forget it" tool that starts saving you money in minutes.

It works by wrapping your existing OpenAI client. Once it's installed, it automatically does three key things:

Smart Model Routing: This is the big one. It analyzes your requests and automatically routes simple tasks to cheaper models (like GPT-3.5) while saving the expensive, powerful models (like GPT-4) for the complex tasks that actually need them.
Prompt Optimization: The tool's AI rewrites your prompts before they hit the OpenAI API to be 40-60% shorter. You get the same (or better) quality results for a fraction of the token cost.
Response Caching: If your application sends the same request multiple times, CostLens will catch it and return the cached result instantly, for free.

How It Works (It's Quick)

I really wanted to make this as easy as possible, with no need to refactor your existing logic.

You just import the client, wrap your existing OpenAI instance, and... that's it. You can keep using the OpenAI client just like you always have, and the optimizations happen in the background.

It looks something like this:

import CostLens from 'costlens';
import OpenAI from 'openai';

// 1. Initialize CostLens
const costlens = new CostLens({
  apiKey: process.env.COSTLENS_KEY
});

// 2. Wrap your existing OpenAI client one time
const openai = costlens.wrapOpenAI(new OpenAI());

// 3. Use it exactly as you did before!
// Savings happen automatically
const result = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [...]
});

// ✅ This request is automatically optimized,
// ✅ routed to the best-priced model,
// ✅ and cached for future use.

I'm Looking for Testers and Feedback!

This started as a personal project, but I think it could be genuinely useful for other indie devs, startups, or anyone who wants to keep their AI experiments affordable.

I just launched a Free plan that lets you optimize up to $100/month of AI spend, so you can try it out without any risk.

If you're a developer using the OpenAI API, I would be incredibly grateful if you'd give it a try and let me know what you think.