Building Robust CI/CD Pipelines for LLM Applications

I recently shipped an automated code review agent that runs inside our CI pipeline. It reads git diffs and flags potential bugs, missing tests, and style issues before a human reviewer opens the pull request. In this tutorial, I will walk through the exact Python script and GitHub Actions workflow I use, powered by Oxlo.ai inference.

What you'll need

• Python 3.10 or newer
• An Oxlo.ai API key from https://portal.oxlo.ai
• A Git repository with at least one commit
• The OpenAI SDK: pip install openai
• A GitHub repository if you want to run the final CI step

Step 1: Scaffold the review script

We start with a single Python file that loads the Oxlo.ai client and accepts a diff via stdin. This keeps the agent stateless and easy to invoke from any CI runner.

import os
import sys
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

def get_diff():
    return sys.stdin.read()

if __name__ == "__main__":
    diff = get_diff()
    if not diff.strip():
        print("No diff provided.")
        sys.exit(0)

Step 2: Write the system prompt

The system prompt is the only configuration the agent needs. I keep it in a separate variable so I can tune it without touching the logic.

SYSTEM_PROMPT = """You are a senior staff engineer performing code review.
Review the provided git diff and output a JSON object with exactly two keys:
- "issues": a list of objects, each with "severity" (critical, warning, or note), "file", "line", and "message".
- "summary": a one-sentence overview of the change.

Be concise. Only flag real problems: logic errors, missing error handling, security risks, or unclear naming. Do not comment on formatting unless it hurts readability."""

Step 3: Call Oxlo.ai with JSON mode

I use Llama 3.3 70B because it follows structured instructions reliably and runs without cold starts on Oxlo.ai. We enable JSON mode and parse the response so the CI runner can act on it.

import json

def review_diff(diff_text: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Review this diff:\n\n{diff_text}"},
        ],
        response_format={"type": "json_object"},
        temperature=0.2,
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

if __name__ == "__main__":
    diff = get_diff()
    result = review_diff(diff)
    print(json.dumps(result, indent=2))

Step 4: Add CI-friendly exit codes

A pipeline step needs to pass or fail. I count critical issues and return a non-zero exit code when any are found, which blocks the merge until a human overrides.

def report_and_exit(result: dict):
    issues = result.get("issues", [])
    critical_count = sum(1 for i in issues if i.get("severity") == "critical")

    for issue in issues:
        icon = {"critical": "❌", "warning": "⚠️", "note": "ℹ️"}.get(issue["severity"], "•")
        print(f"{icon} [{issue['severity'].upper()}] {issue['file']}:{issue.get('line', '?')} - {issue['message']}")

    print(f"\nSummary: {result.get('summary', 'No summary provided.')}")
    print(f"Found {critical_count} critical issue(s).")

    if critical_count > 0:
        sys.exit(1)
    sys.exit(0)

if __name__ == "__main__":
    diff = get_diff()
    result = review_diff(diff)
    report_and_exit(result)

Step 5: Containerize for reproducible CI runs

CI runners should not depend on the host Python environment. A minimal Dockerfile lets us pin the OpenAI SDK version and run the same image locally and in the cloud.

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY review.py .
ENTRYPOINT ["python", "review.py"]

Save this requirements file in the same folder.

openai>=1.0

Build and test locally before pushing.

docker build -t llm-review-agent .
git diff HEAD~1 | docker run --rm -e OXLO_API_KEY=$OXLO_API_KEY -i llm-review-agent

Step 6: Wire into a GitHub Actions workflow

The final piece is a workflow that triggers on pull requests, feeds the diff to the Oxlo.ai-powered agent, and posts the results inline. Because Oxlo.ai uses flat per-request pricing, the cost of reviewing large diffs is predictable, which matters when this runs on every push.

name: LLM Code Review

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Build review agent
        run: docker build -t llm-review-agent .

      - name: Run Oxlo.ai review on PR diff
        env:
          OXLO_API_KEY: ${{ secrets.OXLO_API_KEY }}
        run: |
          git diff origin/${{ github.base_ref }}...HEAD | \
            docker run --rm -e OXLO_API_KEY -i llm-review-agent

Run it

Here is the complete review.py assembled from the steps above. Export your Oxlo.ai key and pipe any git diff into it.

import os
import sys
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

SYSTEM_PROMPT = """You are a senior staff engineer performing code review.
Review the provided git diff and output a JSON object with exactly two keys:
- "issues": a list of objects, each with "severity" (critical, warning, or note), "file", "line", and "message".
- "summary": a one-sentence overview of the change.

Be concise. Only flag real problems: logic errors, missing error handling, security risks, or unclear naming. Do not comment on formatting unless it hurts readability."""

def get_diff():
    return sys.stdin.read()

def review_diff(diff_text: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Review this diff:\n\n{diff_text}"},
        ],
        response_format={"type": "json_object"},
        temperature=0.2,
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

def report_and_exit(result: dict):
    issues = result.get("issues", [])
    critical_count = sum(1 for i in issues if i.get("severity") == "critical")

    for issue in issues:
        icon = {"critical": "❌", "warning": "⚠️", "note": "ℹ️"}.get(issue["severity"], "•")
        print(f"{icon} [{issue['severity'].upper()}] {issue['file']}:{issue.get('line', '?')} - {issue['message']}")

    print(f"\nSummary: {result.get('summary', 'No summary provided.')}")
    print(f"Found {critical_count} critical issue(s).")

    if critical_count > 0:
        sys.exit(1)
    sys.exit(0)

if __name__ == "__main__":
    diff = get_diff()
    if not diff.strip():
        print("No diff provided.")
        sys.exit(0)
    result = review_diff(diff)
    report_and_exit(result)

Test it against the last commit.

export OXLO_API_KEY="sk-oxlo.ai-..."
git diff HEAD~1 | python review.py

Example output from a real review.

⚠️ [WARNING] auth.py:42 - Hardcoded timeout may cause flaky tests under high load.
ℹ️ [NOTE] auth.py:55 - Consider renaming `do_thing` to `validate_token`.

Summary: Adds bearer token validation to the auth middleware but introduces a hardcoded timeout.
Found 0 critical issue(s).

Wrap-up

You can extend this agent by splitting large diffs into file chunks and calling Oxlo.ai in parallel. The flat per-request pricing means fanning out across ten files costs the same as one, which keeps the pipeline economical. Another solid next step is to cache results in Redis keyed by commit SHA so reruns of identical diffs do not burn requests.