DEV Community

shashank ms
shashank ms

Posted on

Using LLMs for Code Analysis: A Step-by-Step Guide

I need a lightweight way to scan Python code for bugs, security risks, and style issues before opening pull requests. In this guide, I will build a CLI tool that sends source files to an LLM and returns a structured JSON report. I run it on Oxlo.ai because their flat per-request pricing lets me pass entire files to the model without counting tokens.

What you'll need

I use the DeepSeek V3.2 model for the analysis because it handles code and reasoning well, and it is available on the Oxlo.ai free tier.

Step 1: Initialize the client and verify the connection

I keep my API key in an environment variable so it does not leak into source control. This first script confirms that the Oxlo.ai endpoint is reachable.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[
        {"role": "user", "content": "Say 'Connection OK'"}
    ],
)

print(response.choices[0].message.content)

Step 2: Lock down the system prompt

The system prompt is the contract. It defines the persona, the categories to check, and the exact JSON schema the model must return. I keep it strict to make parsing trivial.

SYSTEM_PROMPT = """You are a senior software engineer performing static analysis on Python code.
Analyze the provided code for the following categories:
1. Bugs: logic errors, off-by-one issues, unhandled exceptions.
2. Security: SQL injection risks, hardcoded secrets, unsafe eval usage.
3. Style: PEP 8 violations, unclear naming, missing type hints.
Return your findings as a JSON object with this exact schema:
{
  "file": "string",
  "summary": "string",
  "issues": [
    {
      "category": "bug|security|style",
      "line": integer or null,
      "severity": "low|medium|high",
      "description": "string",
      "suggestion": "string"
    }
  ]
}
If no issues are found, return an empty issues array. Output only the JSON object, with no markdown formatting."""

Step 3: Read a file and send it to the model

This function slurps the file, wraps it in a markdown code block for clarity, and sends it to DeepSeek V3.2 on Oxlo.ai. Because Oxlo.ai charges per request rather than per token, I do not need to worry about the file length driving up cost.

import json
from pathlib import Path

def analyze_file(file_path: str) -> dict:
    path = Path(file_path)
    code = path.read_text(encoding="utf-8")

    user_message = f"Analyze the following Python file: {path.name}\n\n

```python\n{code}\n```

"

    response = client.chat.completions.create(
        model="deepseek-v3.2",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )

    raw = response.choices[0].message.content.strip()

    # Strip markdown fences if the model returns them despite instructions
    if raw.startswith("

```json"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()
    elif raw.startswith("

```"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()

    return json.loads(raw)

Step 4: Add error handling and pretty printing

Network calls fail and models occasionally hallucinate syntax. I wrap the parser in a try block and print a human readable report so I can scan results quickly.

def print_report(result: dict):
    print(f"File: {result.get('file', 'unknown')}")
    print(f"Summary: {result.get('summary', 'N/A')}")
    print("-" * 40)

    issues = result.get("issues", [])
    if not issues:
        print("No issues found.")
        return

    for issue in issues:
        line_info = f"line {issue['line']}" if issue.get("line") else "general"
        print(f"[{issue['severity'].upper()}] {issue['category']} ({line_info})")
        print(f"  Description: {issue['description']}")
        print(f"  Suggestion:  {issue['suggestion']}")
        print()

def safe_analyze(file_path: str):
    try:
        result = analyze_file(file_path)
        print_report(result)
    except Exception as e:
        print(f"Analysis failed for {file_path}: {e}")

Step 5: Batch process a directory

Most codebases have more than one file. I glob for Python files recursively and run the analysis on each. With Oxlo.ai, the cost for the batch is predictable because each file is exactly one request.

def analyze_directory(directory: str, pattern: str = "*.py"):
    paths = list(Path(directory).rglob(pattern))
    print(f"Found {len(paths)} files to analyze.\n")

    for p in paths:
        print(f"Analyzing {p} ...")
        safe_analyze(str(p))
        print()

Run it

I create a deliberately buggy file named example.py and run the analyzer.

# example.py
import os

def calculate_average(numbers):
    total = sum(numbers)
    return total / len(numbers)  # ZeroDivisionError risk

password = "hardcoded_secret_123"

eval(input("Enter command: "))

Then I call the analyzer from the CLI.

if __name__ == "__main__":
    safe_analyze("example.py")

The output looks like this.

File: example.py
Summary: Found 3 issues: a potential runtime bug, a hardcoded secret, and unsafe use of eval.
----------------------------------------
[HIGH] bug (line 5)
  Description: Division by zero when numbers is an empty list.
  Suggestion: Check if len(numbers) == 0 before dividing, or return 0.0.

[HIGH] security (line 8)
  Description: Hardcoded password detected in source code.
  Suggestion: Load credentials from environment variables or a secrets manager.

[HIGH] security (line 10)
  Description: User input is passed directly to eval, allowing arbitrary code execution.
  Suggestion: Use ast.literal_eval for safe evaluation, or avoid eval entirely.

Wrap-up and next steps

This tool gives me a fast, automated second pair of eyes on every commit. Two concrete ways to extend it:

  1. Wire the script into a pre-commit hook so modified files are scanned automatically before every push.
  2. Modify the system prompt to request a unified diff for each suggestion, then feed that diff back to the model or apply it with patch to auto-fix simple issues.

Top comments (0)