🛑 Stop Wasting API Calls: How to Build a Dead-Simple Caching Layer for AI Apps

Building AI applications is incredibly fun right up until you check your API dashboard and realize you've been burning through credits by sending the exact same prompts during development and testing.

Whether you are using Claude, Gemini, or OpenAI, rate limits and latency are real bottlenecks. If you are building wrappers, agents, or generation tools, you need a caching layer.

Here is a highly effective, zero-dependency caching wrapper in TypeScript that you can drop into any project today.

The Concept

Instead of calling the LLM directly, we pass the prompt through a caching function. We hash the prompt (or use it as a key) and check if we already have a response stored. If yes, we return the cached string instantly. If no, we make the expensive network call, save the result, and return it.

The Code (TypeScript)

This uses a simple in-memory Map, which is perfect for local development or single-instance edge functions.

typescript
// Define an in-memory cache
const responseCache = new Map<string, string>();

async function fetchWithCache(prompt: string): Promise<string> {
  // 1. Check if we already have the exact prompt cached
  if (responseCache.has(prompt)) {
    console.log("⚡ Returning from Cache (0ms)");
    return responseCache.get(prompt)!;
  }

  // 2. If not, make the actual API call (using a generic fetch as an example)
  console.log("☁️ Fetching from API...");
  const response = await fetch("[https://api.your-llm-provider.com/v1/generate](https://api.your-llm-provider.com/v1/generate)", {
    method: "POST",
    headers: { "Authorization": `Bearer ${process.env.API_KEY}` },
    body: JSON.stringify({ prompt })
  });

  const data = await response.json();
  const textResult = data.choices[0].text;

  // 3. Store the result in the cache for next time
  responseCache.set(prompt, textResult);

  return textResult;
}