DEV Community: Shunya Shida

Why My Cache Has No TTL

Shunya Shida — Mon, 03 Aug 2026 14:14:35 +0000

Last time, I wrote about the dilemma of feeding specs scattered across multiple repositories to AI (Index Everything, or Read Everything?). This post is a devlog for the caching layer of the service I mentioned there (Repospec).

My answer in that post was "outsource just the search" — which means the spec files live in PostgreSQL.

In other words, PostgreSQL holds a cache of files that have already been pushed to GitHub. And that cache has no TTL. Once a file goes in, it stays. Indefinitely, by design.

If you've ever built a cache, this probably makes your skin crawl. It made mine crawl too — the first version absolutely had a TTL. This post is my personal record of how I talked myself out of it.

"It's a cache, so... Redis, right?"

I'll be honest: a lot of this was figured out as I went, not planned up front.

The one thing I knew from the start was that fetching from GitHub on every request was a bad idea, purely for speed (the previous post covers the broader reasoning). So caching was always part of the plan. And my thought process for choosing the store was, in its entirety: "It's a cache, so... Redis, right?" Not my proudest engineering decision, but I suspect it's the same reflex most of us have.

The TTL went in just as automatically. Files fetched from GitHub would expire after a fixed window. Why? Because that's what caches do. That was the whole reason.

I forgot about search

Here's the embarrassing part. The entire point of this service is "find the spec you can't remember where you wrote."

Search is not a feature — it's the product.

And yet there I was, picking a cache store before thinking about how search would work at all. I got absorbed in the architecture and drifted away from the actual problem.

Once search finally got my attention, the answer more or less picked itself: query PostgreSQL directly. Redis is excellent at being fast and excellent at being a cache, but for the kind of fine-grained text search I needed, Postgres was simply the better fit for this use case.

Wait — do I actually need to delete anything?

With the store settled, I looked at the TTL again. And I stalled.

Why am I deleting this data at all?

I tried putting into words what this cache actually needed to guarantee, and it came out like this:

What matters is staying in sync with the latest data on GitHub — not that cache entries get deleted on some appropriate schedule.

A TTL doesn't guarantee freshness. It just guarantees deletion. Sure, the entry gets re-fetched afterward and ends up fresh — but "eventually re-fetched" is not the same thing as "in sync." Worse, in this service, a file that expires from the cache also disappears from search results. "I searched for it and it wasn't there" is the single failure mode I refuse to accept.

So the TTL went.

There's a flip side, though. Removing the TTL means holding onto files from people's private repositories indefinitely. So I also decided what not to hold: the search logs store no spec content and no snippets — queries only. I'd call it less a privacy stance and more a "this is confidential material and I should act like it" stance.

Why not both?

Of course I considered the two-store setup: Redis for caching, PostgreSQL for search. It's a perfectly reasonable architecture. I just couldn't find a benefit that applied here.

The thing is, this cache doesn't behave like a network cache. A network cache grows without bound unless you evict — eviction is load-bearing. This cache has a natural ceiling: it can never hold more than the spec files users have registered. Nothing else ever gets in. The thing a TTL usually protects you from simply doesn't exist here.

The second reason is plumbing. With two stores, every GitHub update has to be synced to both Redis and Postgres. Two sync targets means two ways to drift. I wanted exactly one.

Does Postgres lose to Redis on raw speed? Yes, clearly. But Postgres offered a much wider range of search capabilities, and this whole thing runs behind MCP — an AI agent reading spec files. On that path, a few tens or hundreds of milliseconds are effectively invisible. Shaving milliseconds off a route where the consumer is an LLM buys you nothing.

Freshness now rides on webhooks

With the TTL gone, freshness has exactly one owner: GitHub's push webhook.

This was a design commitment from day one: keeping the cache fresh must never be a human's job. Imagine pushing a commit, then having to open a dashboard and click a "re-sync" button. That's real friction, and I know myself — I wouldn't do it. With webhooks, there's a small lag, but the chore simply doesn't exist.

That said, the implementation turned out to be a lot more than "receive notification, update row"...
That story deserves its own post.

Search is a deeper rabbit hole than I expected

The webhook wasn't the only bill that arrived later. The other one was chunking.

The first version stuffed each spec file into a single column, whole. That fit my usual approach — get something working, then improve it — so in it went. But a column holding a massive blob of text is not a healthy thing to run search queries against. The files needed to be split into reasonably sized pieces.

Then the questions started. What granularity actually gives the best search precision and usability? What about files without clean structural boundaries, unlike Markdown? Turns out "just split the text" is not a thing — I only learned that by trying. This rabbit hole goes as deep as you're willing to dig, so for now the chunk size is a hardcoded guess. There's a very real chance this ends up delegated to something like Elasticsearch eventually, but I'd rather tune it against real usage than against my imagination.

When you should NOT copy this

Let me be clear about where this setup does not belong.

It only works because two preconditions happen to hold:

The cached data is finite. There's a hard ceiling — "however many spec files got registered" — so I can get away with never evicting. If your data grows without bound, you need eviction, which means you need a TTL (or something like it).
Latency requirements are loose. On a path where an AI agent reads spec files, a few hundred milliseconds might as well be zero. If your path is measured in milliseconds, Postgres-as-cache is not an option.

If either one fails for you, just use Redis. It's the right default for a reason.

And to be clear, this post says nothing about network-level caching in general — there are far better articles on that. Everything here only applies because the data had an unusual shape and MCP made latency nearly irrelevant.

"Cache means Redis" is a reflex most of us carry. It's a good reflex. But depending on what you're actually protecting, a setup like this one can be enough.

I'd genuinely like to hear how others have reasoned about this. Have you ever dropped a TTL — or deliberately kept one where it wasn't strictly needed? And if you've dealt with chunking for search, I'd love to know where you landed on granularity. That part is still a guess on my side.

Originally written in Japanese (https://zenn.dev/shida_dev/articles/a4dfcec6c08ad5). Translated with AI assistance and reviewed by the author.

Index Everything, or Read Everything? The Dilemma of Feeding Specs to AI in Multi-Repo Development

Shunya Shida — Sun, 19 Jul 2026 02:30:25 +0000

The specs exist. The AI just can't see them.

I've always been the type who builds hobby projects, gets satisfied halfway through, and never actually finishes. For a long time I wanted to change that — to build something all the way to the end and actually ship it. So a bit over a year ago, I started building small, microservice-style pieces: if each piece is small, I can actually finish it, reuse it later, and it won't eat months of my life. It might sound like overkill for a solo project, but for me it was a trick to "cut things down to a finishable size."

Then AI-assisted development became the norm. To give the AI context, I started writing proper spec documents even for personal projects. So far, so good.

The problem: with microservices, my repos are separate. The repo I have open in Cursor right now is Service A. The spec I want to reference lives in Service B's repo. The spec exists. It's written down. The AI just can't see it.

"How do I hand the AI a spec that lives outside the workspace I currently have open?" This article is a record of me taking far longer than I needed to turn that problem over in my head.

The actual dilemma: two options, both with a price tag

When you boil it down, there are really only two options (probably).

Option A: Add it to the workspace (index everything)

Use Cursor's multi-root workspaces and add the entire spec repo to your workspace. I actually tried this.

It wasn't terrible, but it wasn't exactly pleasant either. Irrelevant files get pulled into context. Files from a repo I'm not even working on show up in search. The symptoms I remember: edits landing in places I didn't ask to change, and token usage ballooning. All I wanted was to read a few spec files, and instead I was paying the tax of putting an entire repo into the AI's field of view.
(AI models keep getting better, so maybe this is less of an issue today.)

To be fair, multi-root workspaces themselves have evolved. In its April 2026 release, Cursor made it so a single agent session can target a multi-folder workspace and make cross-repo changes (changelog). For "fix something across the frontend and the backend" workflows, it's genuinely better now. But the structure hasn't changed: every folder you open becomes part of the index. For "I just want to read a few spec files," it's still overkill.

Option B: Don't add it — fetch on demand (read everything)

Keep the repo out of your workspace and have the AI go fetch files when needed. For example, by writing the paths into your rules file.

If you know the file path, this works fine. The problem is the "wait, I know I wrote this down somewhere" case. Since the AI doesn't know the path, it has to read through the candidate files to find it. Which means loading full file contents into context just to run a search. That's a lot of tokens.

So the dilemma is:

Pay in noise to index everything, or pay in tokens to read everything.

The options I considered — and my false assumption

"If it doesn't exist, build it" is a fine spirit, but if there's a giant's shoulder to stand on, you should stand on it. Before jumping into building my own thing, I did look at existing solutions.

Self-hosted docs tools: I found there were tools that let you run a GitBook-like setup locally with Docker. But they needed customization to fit my workflow, felt inflexible, and didn't offer the polished experience of the paid tools. Which led me to the classic thought: "If it's not that hard to build, maybe I'll just build it myself?" (Ah yes, the person who was going to build it no matter what.)

External SaaS: Documentation-management services were either shockingly expensive or way too much for a solo project's needs.

GitHub MCP Server: In my research chats with an AI at the time, there was a line saying it "needs to run locally," and based on that I ruled it out from the start. If I have to run a container locally anyway, I figured, it's no different from the self-hosted option.

— And here's where I have to make an embarrassing confession. That exclusion was based on a false assumption. GitHub MCP Server has a remote-hosted version, and it went GA in September 2025 (GitHub Changelog). GitHub hosts it, you authenticate once with OAuth, and that's it. No Docker, no PAT management. I only learned this while researching for this very article.

In other words: if I had been a little more skeptical of the AI's research back then and gone to the primary sources, there's a good chance I never would have started building this product. I'm always telling my nephew not to blindly trust what AI says, and yet here we are...
AI research can be based on outdated or plain wrong premises — I re-learned that lesson only after building an entire product on top of it. That said, I still think it was a good experience (I know how that sounds, but as I'll explain in the next section, what I built didn't turn out to be pointless either).

The problem that remained: search that never feels quite right

The remote GitHub MCP Server is genuinely capable. It can fetch files from private repos without a local clone. Honestly, if you know the file path, listing your spec paths in a rules file and letting GitHub MCP read them solves most of my use case, for free.

What about search, then? Turns out that works too. GitHub MCP Server has a code search tool, and GitHub's code search indexes the private repos you have access to (official docs). If I wrote "you can't do cross-repo search with GitHub MCP," that would simply be false.

But when I map it onto the daily "where did I write that part of the spec again?" use case, a few things snag. All of these are documented behavior (official docs):

Only the default branch is searched, and only files under 384 KB. Specs usually fit, but if you keep spec variants on different branches, those are out of scope.
The code search API is rate-limited to 10 requests per minute, even authenticated. That's a separate, much tighter bucket than the regular API (5,000/hour). AI agents love to fire off search queries in trial-and-error loops, so you feel this limit quickly.
Search results are mostly file metadata. (The API can hand back short highlight fragments if you ask for them, but that's about it.) To actually read the content, you need a follow-up fetch. So it's a minimum of two round trips: search, then fetch.

None of these are "it doesn't work." They're all "it never feels quite right for daily use." But "I know I wrote this somewhere" happens multiple times a day. Do I really want to step on that friction every single time?

My answer was to hand search off to a search engine (of sorts). Put the spec contents in PostgreSQL and do the search in a single DB query (eventually vector search too). Return to the AI only a snippet: the matching lines plus a few lines of context. No punishing rate limits, no extra round trips, and the only thing landing in the AI's context is the fragment it actually needs. I'm still mid-development, so it's too early to make big claims about the results — but at minimum, the "now where did I write that" hunts seem to be going away.

What I'm building: a small, read-only window onto your specs

That's what led to Repospec, which I'm building now. You register "this file in this repo is a spec" on a dashboard, and then from Cursor or Claude Code, via MCP, you get fetch and cross-repo search over just your registered specs. (Note: it's still pre-launch — only a landing page for now, where you can sign up for release notifications.)

The design principle was to strip away everything unnecessary and keep it simple. Which means there are things I deliberately did not build: editing features and version control, among others. I wanted to stay focused on my own actual problem, and if this was going to be a product, I did not want it to become "a worse GitBook." The source of truth stays in your GitHub repos; this thing is strictly read-only plumbing.

Do you actually need this?

Write your spec repo paths into .cursor/rules and let GitHub MCP Server read them. If that covers you, that's completely fine. If your project is small enough that you can remember your file paths, you don't need another tool.

This dilemma only really starts to hurt when "multiple repos" × "I can't remember where I wrote what" becomes your daily reality. At that point you have three choices: index everything, read everything, or move just the search outside. I wrote this article because I think there's value in simply knowing the options exist.

When it comes to actually "mastering" AI-assisted development, I feel like there's still a lot of room for craft. The itch is never quite scratched. But what I've gained from building this — that's turning out to be not so small.

Originally written in Japanese (https://zenn.dev/shida_dev/articles/8304bc8dc63b10). Translated with AI assistance and reviewed by the author.