A Case Study: My Free-Tier Bot Kept Forgetting the Schema

Last week I wanted a bot that could answer questions about my class project's database. I had a free model endpoint, a free server, and a schema I was tired of reading. I assumed the model would be the weak link. It wasn't.

I used MonkeyCode's free model tier and its free server option to run this experiment. MonkeyCode is an open-source project, and those two offers are what made the whole thing possible. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The database is a small SQLite file with six tables: users, sessions, posts, comments, votes, and a logging table. I wanted to ask questions like "which table tracks logins?" without opening the schema. A natural-language bot seemed like the perfect student project: small, useful, and easy to break.

Background

The idea was to build a tiny retrieval-augmented generation pipeline. That is a fancy way of saying: grab a few relevant pieces of text, stuff them into a prompt, and let a model answer from those pieces. No vector database. No embeddings. Just a SQLite file, a free model endpoint, and a naive retrieval function.

The learning question was simple. Does retrieval quality matter more than model size when the model costs nothing? I suspected the answer was yes. I was right, but not for the reason I expected.

Goal

The goal was to ask natural-language questions about the schema and get correct table names back. The bot did not need to know anything about the data inside the tables. It only needed to know the structure. That felt easy enough for one evening.

Before you run this, you need Python 3.9 or newer, a SQLite database, and an endpoint that speaks the OpenAI chat format. I used MonkeyCode's free model tier for the endpoint and its free server option to host the script. The exact limits change, so check the current docs before you build anything serious.

Implementation

Step one was reading the schema. SQLite stores table definitions in sqlite_master, so a few lines of Python did the job.

import sqlite3

def load_schema(db_path):
    conn = sqlite3.connect(db_path)
    rows = conn.execute(
        "SELECT sql FROM sqlite_master WHERE type='table' AND sql IS NOT NULL"
    ).fetchall()
    conn.close()
    return [r[0] for r in rows]

Step two was retrieval. I skipped embeddings because I wanted to see how far naive keyword matching could go. Each table definition became a chunk, and I scored chunks by how many words from the question appeared in them.

def retrieve(chunks, question):
    scored = []
    for chunk in chunks:
        words = set(question.lower().split())
        score = sum(1 for w in words if w in chunk.lower())
        scored.append((score, chunk))
    scored.sort(reverse=True)
    return [c for s, c in scored if s > 0][:2]

Step three was the prompt. I told the model to answer only from the schema, set temperature to zero, and sent the request to the endpoint. The full script uses only the standard library, so it runs anywhere Python runs.

import json
from urllib import request

MODEL = "your-free-model-id"
URL = "https://your-endpoint/v1/chat/completions"

def ask(question, context):
    prompt = f"Answer using only this schema:\n{context}\n\nQuestion: {question}"
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "temperature": 0,
    }).encode()
    req = request.Request(URL, data=body, headers={"Content-Type": "application/json"})
    with request.urlopen(req) as resp:
        data = json.load(resp)
    return data["choices"][0]["message"]["content"]

Results

When I ran it with "Which table tracks user logins?" it returned:

The user_logins table tracks user logins.

Good. The free model understood the schema. Then I asked a slightly different question: "Where do I find the user?" The script retrieved the sessions table because the word "user" appeared in a comment inside that chunk. The model dutifully answered that sessions stores user IDs. Technically correct, completely useless.

Here are the three questions I ran:

Q: Which table tracks user logins?
A: user_logins

Q: Where do I find the user?
A: sessions   # wrong

Q: How many sessions yesterday?
A: I cannot answer from the schema alone.

The third one is interesting. The model refused to answer because the schema has no data. That is the behavior I actually want. A free model that says "I cannot answer" is more useful than one that invents a table.

Why did the second question fail? The keyword scorer had no notion of meaning. "User" is everywhere. The model could not see the chunks I did not retrieve, so it had no chance to correct the mistake. That is the real lesson: a free model will confidently answer from whatever context you hand it. Garbage retrieval, confident garbage.

Lessons Learned

What did I learn? First, retrieval quality matters more than model size. Second, a free token allowance changes how you design prompts: you start thinking about how many chunks to include before you spend a single token. Third, a free server is fine for experiments, but it is not a production promise. I did not measure latency, uptime, or accuracy. I do not have benchmarks, and I will not pretend otherwise.

The most common mistake is asking the schema questions it cannot answer. "How many sessions yesterday?" is a data question, not a schema question. The bot cannot know that. Another mistake is treating comments inside the schema as retrieval signals. My sessions table had a comment with the word "user", and that single comment hijacked the answer.

If I built it again, I would make the retrieval return the table name as a separate field, and I would ask the model to quote the exact table name before answering. That small change turns a confident hallucination into a checkable statement.

Who Should Not Use This

Who should not use this approach? Anyone who needs guaranteed uptime, low latency, or private data. Free tiers are for learning, not for SLAs. If your project depends on a free server, you have already accepted a risk you do not control. The same goes for the free token allowance: it is enough for experiments, but the numbers change, and you should not build a business on a number you do not own.

Try It

If you want to try it, change the retrieval function to require an exact table name match, or add synonyms. Then ask the same ambiguous question and see if the answer changes. Better yet, find a question that breaks your retrieval and post it somewhere. That failure is the part worth sharing.

If you run this with your own schema, I would love to hear which question broke your bot first. That is the real case study.