LLM and DevOps: Integrating Large Language Models into DevOps Workflows

#engineering #oxlo #ai

We are going to build a deployment triage agent that reads raw CI/CD or Kubernetes logs and returns a structured JSON diagnosis. This saves time during incidents by letting an LLM do the initial log reading and severity classification before a human opens the dashboard. The whole thing runs against Oxlo.ai using the standard OpenAI SDK, so it drops into existing Python tooling without new dependencies.

What you'll need

Python 3.10 or newer
The OpenAI Python SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Sign up for the Free plan if you want to test without a credit card. Oxlo.ai uses flat per-request pricing, which keeps costs predictable even when you feed long stack traces into the model. See https://oxlo.ai/pricing for details.

Step 1: Bootstrap the Oxlo.ai client

First, I verify that I can reach the Oxlo.ai API and that my key is working. I use the standard OpenAI SDK and point it at the Oxlo.ai base URL.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Say OK"}],
)

print(response.choices[0].message.content)

Step 2: Write the DevOps system prompt

The system prompt is the only manual tuning I do. It tells the model how to format output and what to look for in infrastructure logs.

SYSTEM_PROMPT = """You are a DevOps triage assistant. Your job is to analyze deployment or application logs and produce a structured diagnosis.

Follow these rules:
1. Classify severity as one of: critical, warning, info.
2. Identify the root cause in one sentence.
3. List up to three concrete remediation steps.
4. Output strictly valid JSON with keys: severity, root_cause, remediation_steps (array), needs_human_escalation (boolean).

Be concise. Do not include markdown formatting inside the JSON."""

Step 3: Ingest and format logs

I write a small helper that reads a log file and wraps it in a clear instruction so the model knows what to analyze. I truncate extremely large files to the last 100 lines to stay well within context limits.

def load_log(path: str, tail: int = 100) -> str:
    with open(path, "r", encoding="utf-8", errors="ignore") as f:
        lines = f.readlines()
    return "".join(lines[-tail:])


def build_user_message(log_path: str) -> str:
    raw = load_log(log_path)
    return f"Analyze the following deployment log and return JSON only.\n\n

```\n{raw}\n```

"

Step 4: Run structured triage

Now I wire the pieces together. I call the Oxlo.ai chat endpoint with JSON mode enabled so the response is guaranteed to be parseable. I use llama-3.3-70b here because it handles instruction following and structured output reliably, but you can swap in qwen-3-32b or kimi-k2.6 if you need deeper reasoning.

import json

def triage(log_path: str) -> dict:
    user_message = build_user_message(log_path)

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.2,
    )

    content = response.choices[0].message.content
    return json.loads(content)


if __name__ == "__main__":
    result = triage("deploy.log")
    print(json.dumps(result, indent=2))

Step 5: Add a CI/CD gate

To make this useful in a pipeline, I turn the script into a small CLI that exits with a non-zero code on critical findings. This lets me block or flag deployments automatically.

import sys
import argparse

def main():
    parser = argparse.ArgumentParser(description="Triage deployment logs via Oxlo.ai")
    parser.add_argument("--log", required=True, help="Path to the log file")
    parser.add_argument("--output", default="triage.json", help="Where to write the JSON report")
    args = parser.parse_args()

    result = triage(args.log)

    with open(args.output, "w", encoding="utf-8") as f:
        json.dump(result, f, indent=2)

    print(f"Triage written to {args.output}")

    if result.get("severity") == "critical" and result.get("needs_human_escalation"):
        print("Critical issue detected. Failing the pipeline.")
        sys.exit(1)

    sys.exit(0)


if __name__ == "__main__":
    main()

Run it

Save a sample Kubernetes crash-loop log as deploy.log and run the agent.

export OXLO_API_KEY="sk-oxlo.ai-..."
python triage.py --log deploy.log --output report.json

Example output written to report.json:

{
  "severity": "critical",
  "root_cause": "The application container is crashing due to a missing DATABASE_URL environment variable.",
  "remediation_steps": [
    "Add the DATABASE_URL secret to the deployment manifest.",
    "Verify the ConfigMap is mounted in the correct namespace.",
    "Restart the deployment and check pod readiness."
  ],
  "needs_human_escalation": false
}

Because Oxlo.ai charges a flat rate per request, running this against a 500-line stack trace costs the same as a one-line summary. That makes it practical to run on every failed build without worrying about token math.

Next steps

Two concrete ways to extend this. First, add function calling so the agent can open a GitHub issue or trigger a rollback via your existing API. Second, switch to deepseek-v3.2 or kimi-k2.6 if you start feeding the agent multi-file build artifacts and need stronger reasoning across long contexts. Both are available on Oxlo.ai with the same request-based pricing and OpenAI-compatible SDK.

DEV Community