LLM for Anomaly Detection: A Guide

#learnai #oxlo #ai

We are going to build a server log anomaly detector that feeds batches of system logs to an LLM and flags suspicious patterns in plain English. This saves DevOps teams from maintaining brittle regex rules. Because Oxlo.ai uses flat per-request pricing, you can pack large log batches into the context window without worrying about input length.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Set up the Oxlo.ai client

I start by importing the OpenAI SDK and pointing it at Oxlo.ai's endpoint. This is a drop-in replacement, so the code looks identical to the standard OpenAI pattern.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Prepare synthetic logs

To make this runnable without a production cluster, I generate a small list of realistic nginx-style logs. Four lines are normal, and two contain anomalies: a SQL injection probe and a brute-force login attempt.

logs = [
    "127.0.0.1 - - [10/Oct/2023:13:55:36 -0700] \"GET /index.html HTTP/1.1\" 200 612",
    "127.0.0.1 - - [10/Oct/2023:13:55:37 -0700] \"GET /about.html HTTP/1.1\" 200 354",
    "192.168.1.45 - - [10/Oct/2023:13:56:01 -0700] \"GET /login.php?id=1' OR '1'='1 HTTP/1.1\" 403 298",
    "127.0.0.1 - - [10/Oct/2023:13:56:12 -0700] \"POST /api/data HTTP/1.1\" 200 89",
    "10.0.0.99 - - [10/Oct/2023:13:56:45 -0700] \"POST /login HTTP/1.1\" 401 312",
    "10.0.0.99 - - [10/Oct/2023:13:56:46 -0700] \"POST /login HTTP/1.1\" 401 312",
    "10.0.0.99 - - [10/Oct/2023:13:56:47 -0700] \"POST /login HTTP/1.1\" 401 312",
    "127.0.0.1 - - [10/Oct/2023:13:57:02 -0700] \"GET /dashboard HTTP/1.1\" 200 1024"
]

Step 3: Define the system prompt

The system prompt tells the model how to behave. I want structured output so I can parse it programmatically. I ask for a JSON list of anomalies with severity and reasoning.

SYSTEM_PROMPT = """You are a precise log anomaly detector. Analyze the provided server logs and identify any suspicious or anomalous entries.

Rules:
- Return ONLY a valid JSON array.
- Each object must have fields: line_index (int), severity (low|medium|high), reason (string).
- If no anomalies are found, return an empty array: [].
- Do not include markdown formatting or explanations outside the JSON."""

Step 4: Build the analysis pipeline

I create a function that joins the logs into a numbered list, sends them to Oxlo.ai, and returns the parsed JSON. I use Llama 3.3 70B because it follows structured instructions reliably.

import json

def detect_anomalies(log_lines):
    numbered = "\n".join(f"{i}: {line}" for i, line in enumerate(log_lines))
    user_message = f"Analyze these server logs and return JSON:\n\n{numbered}"

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.1,
        max_tokens=1024
    )

    content = response.choices[0].message.content
    content = content.strip().removeprefix("

```json").removeprefix("```

").removesuffix("

```

").strip()
    return json.loads(content)

Step 5: Add alerting logic

Raw detection is useful, but I want actionable output. I wrap the detector in a small script that prints human-readable alerts for any high-severity findings.

anomalies = detect_anomalies(logs)

for finding in anomalies:
    idx = finding["line_index"]
    severity = finding["severity"]
    reason = finding["reason"]
    log_line = logs[idx]

    if severity == "high":
        print(f"[ALERT] High severity anomaly at line {idx}")
        print(f"  Log: {log_line}")
        print(f"  Reason: {reason}\n")
    else:
        print(f"[INFO] {severity} anomaly at line {idx}: {reason}")

Run it

With everything wired together, running the script produces the following output. The model correctly flags the SQL injection probe and the repeated failed logins.

$ python anomaly_detector.py

[ALERT] High severity anomaly at line 2
  Log: 192.168.1.45 - - [10/Oct/2023:13:56:01 -0700] "GET /login.php?id=1' OR '1'='1 HTTP/1.1" 403 298
  Reason: SQL injection attempt detected in URL parameter

[INFO] medium anomaly at line 4: Repeated failed login attempts from same IP
[INFO] medium anomaly at line 5: Repeated failed login attempts from same IP
[INFO] medium anomaly at line 6: Repeated failed login attempts from same IP

Next steps

Wire the script to a live log source such as tail -f on a local file or a CloudWatch log stream, and process sliding windows of 50 lines at a time. If you need deeper reasoning across multi-turn sessions, swap in kimi-k2.6 or deepseek-v3.2 from Oxlo.ai's model catalog.

DEV Community