The Cache Returned Yesterday's Answer, and the Model Wasn't Wrong

Have you ever cached an LLM response to save money and then served a stale answer the next day? I did, and the confusing part was that the model had done exactly what I asked. The problem was that I asked it the same question without telling it what day it was, and my cache key ignored time entirely. The free model on MonkeyCode's free server made this failure cheap to reproduce, and the fix was a lesson in cache key design.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The cache that saved money and caused the incident

Every morning, a scheduled job on MonkeyCode's free server pulled the previous day's support tickets and asked a free model to write a one-paragraph summary. The summary fed a dashboard that the support team checked before starting their shift, and the whole pipeline cost nothing except a few minutes of compute.

To avoid paying for repeated calls, I added a simple cache: the prompt string became the key, and the model's response was the value. In tests, the same prompt returned the same response, so the cache hit every time and the dashboard loaded instantly. I felt clever about the cost savings, and then the second day arrived.

The dashboard still showed the summary from the first day. The date in the text was wrong, the numbers were wrong, and the support team assumed the pipeline had broken. It hadn't. The cache had done its job perfectly, and that was exactly the problem.

The clue: a date that didn't match

My first instinct was to blame the model, so I checked the logs and found no model call at all. The cache had served the response, and the log line said cache hit with a timestamp from the previous morning. The prompt was identical because I had written it without any reference to the actual date.

"Summarize the support tickets from yesterday" is a prompt that only makes sense if the model knows what today is. The model did not know, because I never told it. The cache key was the prompt string, and since the prompt never changed, the key never changed, and the stale response lived forever.

The reproducible test: send the same prompt on two days

The fix started with a tiny script that proved the problem in seconds. I simulated two days by generating a prompt, hashing it, and comparing the keys.

import hashlib
import datetime

def cache_key(prompt: str) -> str:
    return hashlib.sha256(prompt.encode()).hexdigest()

prompt = "Summarize the support tickets from yesterday."
key_day_1 = cache_key(prompt)
key_day_2 = cache_key(prompt)

print(key_day_1 == key_day_2)  # True

The same key on two different days meant the second day would always receive the first day's summary. The model was never consulted, so it never had a chance to correct the date.

The fix: make time part of the cache key and the prompt

The solution had two parts, and both were necessary. First, I injected the actual date into the prompt so the model knew which day "yesterday" referred to. Second, I included the current date in the cache key so a new day always produced a new key.

def cache_key_with_date(prompt: str, today: str) -> str:
    return hashlib.sha256(f"{prompt}|{today}".encode()).hexdigest()

today = datetime.date.today().isoformat()
yesterday = (datetime.date.today() - datetime.timedelta(days=1)).isoformat()
prompt = f"Summarize the support tickets from {yesterday}."
key = cache_key_with_date(prompt, today)

Now the prompt says "Summarize the support tickets from 2026-08-21," and the key changes every day because the date is part of both. The cache still works for repeated views within the same day, but it can never leak a response across days.

A checklist for caching time-sensitive LLM output

If you cache any model response that might contain a date, time, or other time-sensitive fact, run through this list before you ship:

1. Identify every time reference in the output. Dates, day names, and phrases like "yesterday" or "last week" are all suspicious.
2. Inject the relevant date into the prompt explicitly. Do not rely on the model's internal knowledge of today's date.
3. Include the date in the cache key. A new day must produce a new key, even if the rest of the prompt is identical.
4. Validate the output before serving it. A simple regex for the expected date can catch a stale response before the user sees it.
5. Set a TTL as a safety net. Even with a date-aware key, a short time-to-live protects against edge cases like timezone confusion.

Limitations and who should skip this

The date-injection fix only matters if your output is time-sensitive. If you are summarizing static documentation or classifying evergreen content, a time-aware cache key adds unnecessary complexity. And if your pipeline requires real-time data, caching at all is the wrong move; you should call the model fresh every time.

This approach also assumes the model follows instructions well enough to use the date you provide. Free models can be inconsistent, so the validation step is not optional. I learned that the hard way when a model ignored the injected date and wrote "yesterday" anyway, and the regex caught it before the dashboard did.

The free model on MonkeyCode's free server was a convenient place to hit this wall, but the lesson applies to any LLM cache. The next time your cache returns something that feels wrong, check whether the prompt actually changed. If it didn't, the problem might be that the world did. Have you ever been bitten by a stale LLM cache? Share your story below.