You Don't Need a New Model Call for Every Commit: A Content-Hash Gate for GitLab CI

Every commit triggers the same review. The diff has not changed. The prompt has not changed. But the pipeline still asks the model to think again.

After ten small pushes, your free quota is gone. You are not paying for better output. You are paying for the same output twice. Sound familiar?

The problem is not the model. It is the pipeline. Every push is treated as a fresh signal, even when nothing that matters changed.

The problem is not caching. It is the cache key.

Most teams cache the wrong thing. They cache by branch name, job name, or pipeline ID. That feels safe. It is not.

A model call is deterministic only when all of its inputs are identical. Change any one of these and the old answer becomes invalid:

• the prompt text
• the system prompt
• the model identifier or version
• the output schema or contract
• parameters such as temperature, stop sequence, or max tokens

If you ignore even one of those, you get a stale answer that looks quiet. Quiet is worse than an error because nobody investigates it.

What I use instead

I use a content hash. It includes every input that affects the output. When the hash matches, I reuse the stored response. When the hash differs, I call the model once and store the new pair.

The schema_version field is deliberate. When you change the expected JSON shape, bump that environment variable. The key changes automatically and the gate cannot reuse an old shape.

Here is a minimal FastAPI gate. This is a teaching sketch, not a hardened deployment.

import json
import os
import sqlite3
from hashlib import sha256

from fastapi import FastAPI
from pydantic import BaseModel, Field

app = FastAPI()
DB = os.environ.get('GATE_DB', 'gate.db')
SCHEMA_VERSION = os.environ.get('SCHEMA_VERSION', 'v1')


def init_db():
    with sqlite3.connect(DB) as db:
        db.execute(
            '''
            create table if not exists responses (
                key text primary key,
                response text not null,
                created_at text default current_timestamp
            )
            '''
        )


init_db()


class ModelCall(BaseModel):
    prompt: str
    system: str = ''
    model: str = 'free-default'
    parameters: dict = Field(default_factory=dict)


def stable_key(call: ModelCall) -> str:
    payload = {
        'schema_version': SCHEMA_VERSION,
        'prompt': call.prompt,
        'system': call.system,
        'model': call.model,
        'parameters': call.parameters,
    }
    encoded = json.dumps(payload, sort_keys=True, separators=(',', ':'), default=str)
    return sha256(encoded.encode('utf-8')).hexdigest()


def cache_get(key):
    with sqlite3.connect(DB) as db:
        row = db.execute(
            'select response from responses where key = ?',
            (key,),
        ).fetchone()
    return json.loads(row[0]) if row else None


def cache_put(key, response):
    with sqlite3.connect(DB) as db:
        db.execute(
            'insert or replace into responses(key, response) values(?, ?)',
            (key, json.dumps(response)),
        )
        db.commit()


def call_free_model(call: ModelCall):
    # Replace this with your real free model route.
    # Keep this isolated so you can swap providers without touching cache logic.
    return {'decision': 'hold', 'reason': 'mock'}


@app.get('/health')
def health():
    return {'ok': True}


@app.post('/call')
def call(call: ModelCall):
    key = stable_key(call)
    cached = cache_get(key)
    if cached is not None:
        return {'hit': True, 'key': key, 'response': cached}
    response = call_free_model(call)
    cache_put(key, response)
    return {'hit': False, 'key': key, 'response': response}

The stable_key function is the whole gate. It serializes the prompt, system message, model identifier, parameters, and schema version into one canonical JSON string. Then it hashes the string. The hash is the cache key.

SQLite is enough for a single-instance gate on a free tier. Move to a real database only if you need many concurrent writers.

Where to run it

MonkeyCode has free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server is a practical place to host the gate. Point your CI job at the gate URL and keep the model call logic behind one HTTP endpoint.

But the design does not depend on any specific server. Run it on a small VPS, a container, or a local test box first.

Wire it into GitLab CI

The CI job does not need to know which model is behind the gate. It only needs a URL, a prompt, and a timeout.

model_review:
  image: python:3.12-slim
  variables:
    GATE_URL: 'https://your-free-server.example'
  script:
    - pip install requests
    - python ci/model_review.py
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

The reviewer stays small.

import json
import os
import subprocess

import requests

GATE_URL = os.environ['GATE_URL']


def get_diff():
    result = subprocess.run(
        ['git', 'diff', 'HEAD^', 'HEAD'],
        text=True,
        capture_output=True,
    )
    return result.stdout.strip()


prompt = {
    'system': 'You review a small CI diff and answer with JSON.',
    'model': 'free-default',
    'prompt': 'Review the staged change: ' + get_diff(),
    'parameters': {'temperature': 0},
}

resp = requests.post(f'{GATE_URL}/call', json=prompt, timeout=30)
resp.raise_for_status()
body = resp.json()
print('cache_hit:', body['hit'])
print(json.dumps(body['response'], indent=2))

The first push misses the cache. The second push with the same diff hits it. The third push with a changed system prompt misses again and stores a new row.

That is the behavior you want. It is also the behavior you should prove before trusting it.

Prove it with a local smoke test

Start the service with pip install fastapi uvicorn requests and uvicorn gate_service:app --reload.

Run this small client twice against the same body:

import requests

gate = 'http://127.0.0.1:8000'
body = {
    'prompt': 'Review this diff',
    'system': 'be strict',
    'model': 'free-default',
}

first = requests.post(f'{gate}/call', json=body, timeout=10).json()
second = requests.post(f'{gate}/call', json=body, timeout=10).json()
print(first['hit'], second['hit'])

• The first call prints False.
• The second call prints True.
• Change system by one word and run it again. The call prints False.

Then check the rows:

sqlite3 gate.db 'select key, created_at from responses;'

You should see two rows, not three. That is a five-minute check. Do not skip it.

Limits you should know

This approach is not a general cache. It is a narrow gate for structured, low-temperature model calls.

• Do not use it for creative output. Identical prompts with a sampling temperature above zero can produce different valid answers.
• Put the real model version in the key when the provider exposes one. Otherwise an unannounced model change can poison the cache silently.
• Verify disk persistence on the free server. Some free tiers keep memory but not disk across restarts.
• Add authentication if prompts contain code from private repositories. Do not send secrets to a shared gate.

A cache hit is not a correctness guarantee. It only means the same inputs were seen before.

Here is a quick decision table:

Situation	Use this gate?
Same diff, structured JSON, low temperature	Yes
Creative text, sampling, or brainstorming	No
Private customer data or secrets	No, or self-host with strict access control
High-risk merge decisions	No, keep deterministic checks and human review

Who should not use this

Skip this if your provider already deduplicates requests. Skip it if every output must be freshly evaluated for audit or compliance reasons. Skip it if the model call is so rare that the extra service costs more effort than the quota it saves.

This gate is for teams that call a free model often enough to notice repeat prompts, but not often enough to need a full inference cache.

Start with one non-critical pipeline. Measure how many requests are cache hits before you scale it. Then decide whether the extra moving part is worth it.

The code above is a starting point. Test it on your own prompts and your own provider before you trust it in a merge request.