Getting Started with Edge AI and LLM: A Beginner's Guide

#learnai #oxlo #ai

We are building an Edge Log Triage Agent that runs on a local gateway and classifies log lines in real time. It handles obvious cases with lightweight local rules and escalates ambiguous events to an LLM via Oxlo.ai. This is for teams who monitor distributed devices and need fast, structured alerts without maintaining their own inference cluster.

What you'll need

Python 3.10 or newer
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK: pip install openai
python-dotenv to keep credentials out of source: pip install python-dotenv

Step 1: Project Setup

Create a working directory, store your Oxlo.ai API key in a .env file, and initialize the OpenAI-compatible client that points to Oxlo.ai.

import os
from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY")
)

Step 2: Local Edge Filter

Before we spend an API request, we use a fast regex heuristic on the device. Critical and trivial keywords are handled locally so only uncertain lines leave the edge.

import re

def local_classify(line: str):
    line_lower = line.lower()
    if re.search(r'\b(error|fatal|panic)\b', line_lower):
        return {"severity": "critical", "source": "local", "action": "page_oncall"}
    if re.search(r'\b(info|debug|trace)\b', line_lower):
        return {"severity": "low", "source": "local", "action": "ignore"}
    return None

Step 3: System Prompt

This prompt shapes the agent into a strict log analyst that returns only JSON. Keeping the instructions tight reduces token bloat on the edge side, and because Oxlo.ai charges per request rather than per token, long prompts do not inflate cost. See https://oxlo.ai/pricing for current plan details.

SYSTEM_PROMPT = '''You are a log triage analyst running on an edge gateway.
Analyze the provided log line and return a single JSON object with exactly these keys:
- severity: one of low, medium, high, critical
- category: one of network, disk, memory, application, security, unknown
- action: one of ignore, investigate, restart_service, page_oncall
Do not include markdown, explanations, or surrounding text.'''

Step 4: LLM Triage with Oxlo.ai

When the local filter returns None, we send the line to Oxlo.ai. We use llama-3.3-70b for low latency and predictable per-request pricing, which makes it practical to add multi-line context when needed.

import json

def llm_classify(line: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Log line: {line}"},
        ],
    )
    raw = response.choices[0].message.content.strip()
    # Defensive parse in case of minor whitespace
    if raw.startswith("

```json"):
        raw = raw.split("```

json")[1].split("

```

")[0].strip()
    return json.loads(raw)

Step 5: Main Loop

Here we read a sample log file line by line, apply the local filter, and fall back to the Oxlo.ai classifier. The script prints structured results that a downstream monitoring system can consume.

def triage_log(file_path: str):
    with open(file_path, "r") as f:
        for line in f:
            line = line.strip()
            if not line:
                continue

            result = local_classify(line)
            if result:
                print(f"[LOCAL]  {result['severity'].upper():8} | {line[:60]}")
                continue

            try:
                result = llm_classify(line)
                print(f"[OXLO]   {result['severity'].upper():8} | {result['category']:12} | {line[:60]}")
            except Exception as e:
                print(f"[ERROR]  FAILED    | {line[:60]} | {e}")

if __name__ == "__main__":
    triage_log("sample.log")

Run it

Create a synthetic log file named sample.log and run the script.

# sample.log
2024-05-20T10:00:00Z INFO  server started successfully
2024-05-20T10:01:12Z WARN  connection pool approaching limit
2024-05-20T10:02:33Z ERROR disk write timeout on /dev/sda1
2024-05-20T10:03:45Z ???   unknown telemetry signature 0x4f2a

$ python edge_triage.py
[LOCAL]  LOW      | 2024-05-20T10:00:00Z INFO  server started successf
[OXLO]   MEDIUM   | network      | 2024-05-20T10:01:12Z WARN  connection pool a
[LOCAL]  CRITICAL | 2024-05-20T10:02:33Z ERROR disk write timeout on
[OXLO]   HIGH     | unknown      | 2024-05-20T10:03:45Z ???   unknown telemetry s

Wrap-up

The hybrid approach keeps latency low for routine events while using Oxlo.ai for the decisions that actually matter. Because Oxlo.ai bills per request, you can stuff in multi-line context or stack traces without watching token meters spin. Two concrete next steps: add a local SQLite cache so identical lines do not trigger duplicate requests, and swap llama-3.3-70b for deepseek-v3.2 or kimi-k2.6 when you need deeper reasoning on complex security anomalies.

DEV Community