Hindsight has been running behind my Hermes agent for nearly half a year, and somewhere along the way I almost forgot it was there.
Old decisions come back when they’re needed, conversations carry over into the next session, and work picks up without the familiar ten minutes of explaining everything again. After a while, you stop noticing that part. You expect the agent to remember because most days it does.
That’s a fairly big change from June, when I wrote that Hindsight might be too much hassle for too little return. At the time, keeping another service running felt like a questionable way to save myself work.
It stayed in the stack anyway, and I’ve been meaning to revisit that comment. Looking through the logs for this article made me realize how far my attention had moved on while memory kept doing its job.
Meanwhile, the models available to handle that background work have become cheaper and more capable. If you’re using Hermes every day and still explaining the same project each time you open a session, I’d put custom memory much higher on your list than another model upgrade.
In This Edition
Where Hindsight fits beside my files and workflow instructions
What changed my mind after the early retrieval problems
How two agents hit a shared limit, and what happened to the failed writes
Whether the extra service is worth running in your own setup
Subscribe to All Agents Considered for practical agent workflows and follow-ups on the setups I keep using.
When Remembering Becomes Normal
There’s a particular kind of friction you get used to with AI. A conversation goes well, you settle a decision, and a few days later you’re explaining the whole thing again because the new session has no useful recollection of it.
You start compensating without thinking about it. Before asking for help, you gather the old context and prepare a small briefing for an assistant you’ve already briefed. Eventually that preparation feels like part of using the tool.
Custom memory takes some of that work off your hands. In my setup, Hindsight keeps useful information from sessions and brings relevant context back into later ones, so a decision has a chance of surviving beyond the conversation where it was made.
The difference shows up in ordinary work. Earlier this month, Hermes picked up a research workflow from June and used the output format we’d settled on then. I’d forgotten that I’d specified it, which made it slightly embarrassing to be reminded by my own agent.
Another session brought back an older decision about article numbering. Hermes applied it and carried on, and there was no reason to stop what I was doing and think about the memory system responsible.
Multiply that kind of thing across a working week and you begin to understand why this follow-up took so long. There’s less to complain about, so your attention goes somewhere else.
The week of logs I checked contained 177 recalls and 183 writes. Memory was being used throughout the week while I was thinking about the work in front of me. Those numbers won’t tell you how useful every result was, but they explain why removing the service now would be noticeable.
Even returning to older work feels different. Asking about something from August gives Hermes a place to start without requiring me to remember which conversation contained it. That’s useful when the reason for a decision never made it into the finished document.
It Took Some Settling In
Anyone who read my vendor-lock-in article might remember this sentence:
I added a 3rd party memory provider called Hindsight, but I might drop it as it’s proven to be too much hassle for too little return for me.
There was a reason for that frustration. A retrieval timeout had made Hermes feel slow, and I spent days blaming the model before finding the delay earlier in the chain. The agent was waiting for memory before it even reached the part I’d been blaming.
Adding a service to reduce repetitive work feels a bit silly when you’re spending your evening diagnosing that service. At that point, going back to a simpler setup was tempting.
But the early trouble stopped defining the experience. Hindsight stayed running while the rest of the stack changed around it, and retrieving old context became something I relied on without making a separate decision to trust it.
My setup is still fairly modest. Hindsight runs in a container on hardware I already own, with a Nemotron model through a GPU provider handling fact extraction. That’s the step that turns a session into information the memory system can keep.
Most settings remain at their defaults, with a limit on how much recalled text comes back into each request. A useful reminder about the current project should fit into the work without dragging a pile of unrelated history along with it.
There are more settings available if you want to spend your weekend experimenting. After the early troubleshooting, I was happy to leave them alone and get back to using Hermes.
Better Models Change the Calculation
Nearly half a year is a long time in AI. The cheaper end of the model market has improved enough that background jobs like this deserve another look, especially if your impression of them comes from an older setup.
You can see the broader direction in Artificial Analysis’s tracking of model capability and inference prices. More capable models keep becoming available at lower price points, giving us better options for work that has to happen repeatedly without turning every request into an expensive event.
Memory is a good use for that progress. Extracting a project decision from a conversation is a narrower job than carrying out the project itself. It needs a model that handles the information reliably, but paying for the most expensive model on every background call is a choice you should have a reason for.
And this work adds up. Each session leaves information to process, even when the visible task was small. Cheaper models make it easier to keep that happening as part of normal use instead of saving memory for the occasions when you remember to ask.
My own setup uses an existing provider account and an existing server, so adding Hindsight didn’t add another memory subscription. It still uses compute and provider capacity, but the extra service has been easy enough to accommodate inside the stack.
This is the direction I wanted when putting together the $30 Hermes setup: affordable components that keep being useful after the initial excitement wears off. The options for putting that kind of setup together have improved since then.
For someone already running Hermes daily, I don’t see a good reason to keep putting off third-party memory because of what it used to cost or how capable the cheaper models used to be.
You’ll still spend some time connecting it and checking the results. But if you’re repeatedly supplying context the agent has already seen, you’re already spending time on memory. You’re doing the retrieval yourself.
Even the Failure Went Unnoticed
The most telling detail in the logs was something nobody in the house had noticed.
Two agents use the same Hindsight deployment, each with its own memory bank. One is mine and the other belongs to a family member, but both use the same GPU provider account and API key for extraction.
One morning in September, both were busy. My agent had also delegated work to several subagents, whose returning results triggered additional recalls during the run.
When both banks needed to extract facts around the same time, the provider returned rate-limit errors. A batch of writes retried four times over roughly seven seconds and failed.
Then, about ninety seconds later, the queue tried again. The batch went through and the operation completed.
By the time I found it in the logs days later, there was nothing left for me to fix in that batch. It had failed and recovered while we carried on using the agents.
After the earlier timeout, I’d expected another memory problem to announce itself by getting in the way of my work. This one left a trace in the logs and was over before either of us noticed.
There’s still a useful detail to take from it. Separate memory banks were sharing the same extraction capacity, so two apparently separate agents could hit the same limit. If you’re running several scheduled jobs, check whether they all reach that provider at once before deciding you need a different model.
For my setup, though, the recovery was the more encouraging part. The queue gave the provider time to accept the work, and the failed batch didn’t need me to intervene. That’s the kind of behavior that makes a background service easier to live with.
Memory Fits Around Your Files
None of this has changed where the working documents live. The current article still belongs in its folder, and the instructions for a recurring task still sit beside that task.
Hindsight helps bring back the conversation around the work. If a decision needs to govern every run, it deserves a place in the instructions where Hermes is told to read it.
That’s why the Mistake Ledger still has a job. A verified fix for the research sorter belongs beside the sorter. Recalling an old conversation is useful, but a rule that must apply every time should be easy to find every time.
Both fit comfortably into the same setup. Files give the work a clear shape, while memory helps the agent pick up details you’d otherwise have to supply again.
You don’t need to turn the whole workspace into a memory experiment to get something useful out of it. Start with the projects you return to often and pay attention to whether the old decisions come back correctly. An agent remembering the wrong version of a decision is something to correct before you rely on it more widely.
Once that part is working, give yourself permission to stop fiddling with it. It’s easy to spend so much time improving the machinery that you never get around to using it.
A Much Easier Recommendation Now
Coming back to Hindsight after nearly half a year, the surprise is how little of the story involves maintaining it.
The early frustration was real, but it no longer describes an ordinary week. Most of the time, Hermes retrieves something useful and the work continues. Even the September incident had finished recovering before I knew it had happened.
With cheaper, more capable models available for the background work, custom memory is also easier to recommend than it was when I started. If Hermes is part of your daily routine, give it a way to carry more of that routine forward.
That’s where my setup has ended up. I open Hermes expecting it to know enough about the project for us to get started, and most days it does. Hindsight is still running on the server, but it’s been a while since it felt like another project I had to look after.
What does your agent still make you explain every time you come back? Tell me in the comments.
Subscribe to All Agents Considered for practical ways to make your agent more useful in everyday work.
Top comments (0)