DEV Community

Charlie Zhu
Charlie Zhu

Posted on

The $0 Memory Layer: A JSON File That Makes a Free LLM Remember

#ai #python #productivity #tutorial

Why does every chatbot forget what you told it yesterday? Because you never gave it a notebook. I built a personal assistant on MonkeyCode that keeps a persistent memory in a JSON file, and it runs on the platform's free models and free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you are tired of re-explaining your project to an AI every single session, this is the cheapest fix I have found.

The pattern solves a real pain: context windows are finite, but your requirements grow. Instead of stuffing thousands of lines into a prompt, you save distilled notes and load only relevant ones on demand. That saves tokens, which matters when you are on a generous but still limited allowance. MonkeyCode currently provides a free tier with 10 million tokens for model calls and one free server instance, so the whole experiment costs zero dollars until you decide to scale.

Here is the core class. It treats memory as a simple list of notes with tags and a score, then persists everything to a local file.

# notebook.py
import json, os, time

class Notebook:
    def __init__(self, path="memory.json"):
        self.path = path
        self.data = self._load()

    def _load(self):
        if os.path.exists(self.path):
            with open(self.path) as f:
                return json.load(f)
        return {"notes": []}

    def _save(self):
        with open(self.path, "w") as f:
            json.dump(self.data, f, indent=2)

    def add_note(self, text, tags=None, score=1.0):
        self.data["notes"].append({
            "text": text,
            "tags": tags or [],
            "score": score,
            "created": time.time()
        })
        self._save()

    def query(self, term, top_k=3):
        results = []
        for note in self.data["notes"]:
            if term.lower() in note["text"].lower() or term in note["tags"]:
                results.append(note)
        return sorted(results, key=lambda n: n["score"], reverse=True)[:top_k]
Enter fullscreen mode Exit fullscreen mode

That gives you the storage primitive, but where does the LLM come in? In my setup, every time I read an article or finish a debugging session, I send a one-sentence summary to the model and ask it to extract three tags and a score. The model returns JSON, I feed that straight into add_note. The free models handle this perfectly because the task is short and the output is structured.

# usage example with a mocked client - swap with MonkeyCode API later
def remember_with_llm(client, raw_text):
    prompt = f"""Extract a concise note (max 15 words), tags, and importance score from this text.
Return JSON only: {{"text": "...", "tags": ["..."], "score": 0.0}}
Text: {raw_text}"""
    response = client.generate(prompt)
    return json.loads(response)
Enter fullscreen mode Exit fullscreen mode

The mock returns whatever you feed it, but the real client hits the free model endpoint described in the MonkeyCode README. I do not repeat the exact URL here because endpoints are versioned; the README updates faster than any blog post. The important lesson is that the interface stays identical, so switching from a fake to a real model changes one line.

How do you decide between a JSON file, SQLite, and a vector database? I made a quick decision table for you.

Approach Best for Costs Scaling ceiling
JSON file Single user, simple queries Free, zero setup seconds per query
SQLite Small team, relational filters Free, more APIs minutes per query
Vector DB Semantic search across long histories More tokens for embedding, infra hours or days

For a personal assistant that answers questions about what you learned last week, a JSON file is enough. For a team trying to recall decisions from a shared knowledge base, SQLite wins. For a production app with thousands of users and fuzzy queries, go vector.

The whole thing deploys to the free server with a tiny Flask wrapper. I set an environment variable for the memory path and expose two routes: POST /remember and GET /ask. The free server sleeps after idle time, but a simple health check wakes it up. This is not a production architecture; it is a way to test an idea before you commit to paid infrastructure.

Who should skip this? Anyone building a multi-tenant product, because file locks will corrupt memory under concurrent writes. Anyone who needs semantic similarity beyond simple tag matching, because my keyword scan misses synonyms. And anyone with strict data residency requirements, because the free server may run in a region you do not control.

Still, for a solo developer learning LLMOps, a JSON file is the perfect starter. You get persistence, you get a cheap way to experiment, and you burn almost none of your token budget. Run the code, swap in your MonkeyCode key, and let the free models do the summarization. After one week you will have a list of notes that actually feels like a brain.

Top comments (0)