Running Self-Hosted AI Code Reviews with Ollama on a Small VPS

AI-assisted code review doesn't necessarily require sending every pull request to a hosted AI service.

For some teams, running the review workflow inside their own infrastructure can be an interesting alternative—particularly when they already operate Gitea, GitLab, GitHub Enterprise, or other self-hosted development infrastructure.

I wanted to see how small such a setup could reasonably be.

The result is a fairly simple architecture:

flowchart LR
    git["Git platform"] -->|Webhook| workflow["AI workflow"]
    workflow -->|Repository context + prompt| ollama["Ollama"]
    ollama --> model["Local coder model"]
    model -->|Review comments| git

There is no per-developer component in this architecture. The main constraints are instead the compute required by the model, the size of the pull requests being reviewed, and how quickly you expect reviews to complete.

For my experiments I use AI-Git-Bot as the workflow layer, Ollama as the local inference server, and an open-weight coder model.

The interesting part isn't really the particular bot, though. It's what becomes possible once the Git workflow and the model runtime are separated.

Why run code review locally?

Hosted AI development tools are convenient, and for many teams they're probably the simplest option.

But they also come with trade-offs.

Pricing commonly scales with the number of developers, while the underlying workload doesn't necessarily do so in the same way. A ten-person team might generate fewer pull requests than a three-person team working on a very active repository.

There is also the question of where source code is processed.

Depending on the organization, sending diffs to an external model provider can require additional security, contractual, or compliance review. In some environments it isn't allowed at all.

A self-hosted setup changes those trade-offs:

flowchart LR
    repo["Repository"] -->|Webhook| workflow["Workflow engine"]
    workflow -->|Prompt + context| model["Local model"]
    model -->|Review result| workflow
    workflow -->|Publish findings| repo

The source code, prompts, model inference, and generated review stay within infrastructure you control.

The downside is equally important: you now operate the infrastructure yourself.

You have to think about memory, model performance, updates, monitoring, and what happens when the model produces a poor review.

So this isn't automatically a better architecture. It's just a different one.

What the workflow layer does

A local LLM alone isn't enough to provide automated code reviews.

Something still needs to:

• react to pull-request events,
• retrieve the diff and repository context,
• construct the prompt,
• call the model,
• interpret the response,
• and publish findings back to the Git platform.

That's the role AI-Git-Bot plays in my setup.

It supports GitHub/GitHub Enterprise, Gitea, GitLab, and Bitbucket Cloud and can react to normal Git events.

For example:

Workflow	Trigger	Result
PR review	PR opened / review requested	Summary and findings
Interactive Q&A	Bot mentioned in a PR	Context-aware answer
Unit-test generation	PR opened	Regression tests
E2E / full-stack QA	PR opened	Test execution and results
Issue triage	Issue opened	Classification or assignment
Issue implementation	Issue assigned to coding agent	Implementation pull request
Documentation sync	PR opened	Documentation changes
i18n coverage	PR opened	Missing translations

Not every workflow makes sense with a small local model.

That distinction turned out to be quite important.

A minimal deployment

For a self-hosted experiment, the basic stack consists of:

• AI-Git-Bot
• PostgreSQL
• Ollama
• an open-weight coder model

A simplified Docker Compose setup looks like this:

services:
  app:
    image: tmseidel/ai-git-bot:latest
    ports:
      - "8080:8080"
    environment:
      SPRING_PROFILES_ACTIVE: docker
      DATABASE_URL: jdbc:postgresql://db:5432/giteabot
      DATABASE_USERNAME: giteabot
      DATABASE_PASSWORD: giteabot
      APP_ENCRYPTION_KEY: change-me
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_DB: giteabot
      POSTGRES_USER: giteabot
      POSTGRES_PASSWORD: giteabot
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U giteabot"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped

  ollama-pull:
    image: ollama/ollama:latest
    entrypoint:
      - sh
      - -c
      - sleep 5 && ollama pull qwen2.5-coder:7b
    environment:
      OLLAMA_HOST: http://ollama:11434
    depends_on:
      - ollama

volumes:
  ollama_data:
  pgdata:

Once Ollama is running, the AI integration points at:

http://ollama:11434

with, for example:

qwen2.5-coder:7b

There are no model API credentials involved because inference happens locally.

How small can the model be?

This was the more interesting question.

A 7B coder model is obviously not going to compete with the largest hosted reasoning models on every task.

But not every Git workflow needs the same capabilities.

I've found a useful distinction between workflows that mostly generate natural language and workflows that need reliable agentic behavior.

Workload	7B class	14–32B class
PR review	Works reasonably well	Better
Explain a change	Works reasonably well	Better
Documentation updates	Often sufficient	Better
Issue classification	Depends on required output structure	Better
Multi-step coding agent	Limited	Much more suitable
Strict structured output	Can be unreliable	More reliable

For a pull-request review, a smaller model can still inspect a diff and produce useful natural-language observations.

Agent workflows are harder.

Consider an issue-to-pull-request workflow:

flowchart TD
    issue["Issue"] --> understand["Understand task"]
    understand --> inspect["Inspect repository"]
    inspect --> choose["Choose files"]
    choose --> edit["Modify code"]
    edit --> test["Run tests"]
    test --> result{"Tests pass?"}
    result -->|Yes| pr["Create pull request"]
    result -->|No| diagnose["Interpret failures"]
    diagnose --> edit

Each additional decision increases the importance of reasoning quality, tool use, structured output, and instruction following.

That's where the difference between a small local model and a larger model becomes much more visible.

A hybrid setup may be more practical

Self-hosting doesn't have to mean that every workflow uses the same local model.

For example:

flowchart LR
    event["Git event"] --> workflow["Workflow"]
    workflow -->|Review task| local["Local 7B model"]
    local --> review["PR review"]
    workflow -->|Agentic task| large["Larger model"]
    large --> agent["Coding agent"]

The inexpensive, frequent tasks can run locally.

The less frequent workflows that need stronger reasoning can use a larger local model or an external provider.

I find this architecture more interesting than treating the model provider as a global application setting.

Different workflows have different requirements.

A PR summary, security review, issue classifier, and autonomous coding agent don't necessarily benefit from the same model.

What about the "$5 VPS"?

This needs a qualification.

The application itself is lightweight. Model inference is not.

A quantized 7B model is only a few gigabytes on disk, but the machine also needs memory for the model runtime, context, operating system, database, and application.

So whether an actual $5/month VPS is sufficient depends heavily on the provider, available RAM, CPU performance, context size, and acceptable latency.

If you already have an 8 GB machine, a small home server, a workstation, or other unused infrastructure, the incremental software cost can be close to zero.

If you're renting infrastructure specifically for inference, you should size and benchmark it rather than assuming the cheapest VPS will provide acceptable performance.

For me, the useful observation isn't really "$5."

It's this:

The cost of a self-hosted review service is primarily tied to the inference infrastructure and workload rather than directly to the number of developers using it.

That's a different scaling model from per-seat SaaS.

Whether it's cheaper depends on your team and infrastructure.

CPU inference is possible, but latency matters

A GPU isn't strictly required for experimenting with a 7B quantized model.

CPU inference works.

But "works" and "feels fast" are different things.

For asynchronous workflows such as pull-request review, latency is often less critical than it would be for an interactive coding assistant.

If a review takes a few minutes after a pull request is opened, that may still be perfectly usable.

For interactive Q&A or agentic coding loops, the same latency becomes much more noticeable because every model call is part of a sequence.

This is another reason I wouldn't use one deployment strategy for every AI-assisted development workflow.

Where I think this architecture makes sense

There are a few situations where I've found self-hosting particularly interesting.

Self-hosted Git platforms

A lot of AI developer tooling starts with GitHub support.

If your organization primarily runs Gitea or another internally hosted Git platform, connecting a generic workflow layer to a local model can be more flexible than trying to fit the environment into a GitHub-centric SaaS product.

Source code shouldn't leave the network

Some teams have contractual, regulatory, or internal-security constraints around source code.

Keeping the complete path

flowchart LR
    git["Git platform"] -->|Event + repository data| workflow["Workflow engine"]
    workflow -->|Prompt + context| model["Local model"]
    model -->|Generated result| workflow
    workflow -->|Comment or change| git

inside the same environment makes that boundary easier to reason about.

It doesn't automatically make the system secure—you still have to secure the infrastructure—but the data flow is much simpler.

Existing compute is available

If an organization already operates machines with sufficient RAM or GPU capacity, running another small inference workload may be inexpensive.

The economics are quite different if hardware has to be purchased specifically for the workload.

What this setup doesn't replace

A self-hosted review workflow isn't a replacement for every AI developer tool.

It isn't code completion.

An IDE assistant helps while code is being written. A Git workflow operates after an event such as opening a pull request.

Those are different places in the development lifecycle.

It also doesn't remove the need for human review.

I treat model-generated findings much like static-analysis findings: potentially useful signals that still need context.

And a smaller local model makes that distinction especially important.

The experiment I find interesting

What started as an attempt to run inexpensive AI code reviews has turned into a broader architectural question for me:

How much AI infrastructure actually needs the largest available model?

Some development workflows clearly benefit from powerful reasoning models.

Others look much more like repeated, bounded classification or transformation tasks.

Once those workloads are separated, the architecture becomes more flexible:

flowchart LR
    git["Git"] --> workflow["Workflow"]
    workflow --> route{"Choose model for task"}
    route --> local["Small local model"]
    route --> large["Large local model"]
    route --> hosted["Hosted reasoning model"]

The workflow chooses the appropriate model for the task rather than treating "the AI" as one monolithic dependency.

That's also where I'm currently experimenting with AI-Git-Bot.

The project is MIT licensed, and the repository contains deployment and Ollama examples if you want to reproduce the setup.

If you've tried running code review on small local models, I'd be especially interested in hearing which model and hardware combination you used—and where you found the quality threshold stopped being acceptable.