LLMs for Decision-Making: A Guide

We are going to build a Decision Matrix Agent that turns an unstructured business or technical question into a scored, reproducible recommendation. It evaluates each option against weighted criteria, ranks them, and then critiques its own top pick so you understand the trade-offs. This is useful for engineering teams who want to replace endless debate with a documented reasoning trail.

What you'll need

Python 3.10 or newer installed locally. The OpenAI SDK, which you can grab with pip install openai. An Oxlo.ai API key from https://portal.oxlo.ai. Oxlo.ai is fully OpenAI-compatible, so the SDK works without any adapters.

Step 1: Set up the client and decision schema

I start by defining plain Python dataclasses for criteria and options so the rest of the script stays typed and readable. Then I initialize the OpenAI client pointing at Oxlo.ai.

import json
import os
from dataclasses import dataclass
from typing import List

from openai import OpenAI

@dataclass
class Criterion:
    name: str
    weight: float
    description: str

@dataclass
class Option:
    name: str
    description: str

OXLO_API_KEY = os.getenv("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key=OXLO_API_KEY)

Step 2: Lock down the system prompt

The system prompt is where we enforce structure. I tell the model exactly how to score, how to weight, and what JSON shape to return. Keeping this prompt version-controlled lets you tune decision quality over time.

SYSTEM_PROMPT = """You are a Decision Matrix Agent. Your job is to evaluate options against weighted criteria and return a structured JSON decision report.

Rules:
1. Evaluate every option against every criterion. Give a score from 1 to 10.
2. Multiply each score by the criterion weight to get a weighted score.
3. Sum weighted scores for each option to produce a final rank.
4. Return strictly valid JSON with no markdown formatting.

Required JSON structure:
{
  "recommendation": "Name of winning option",
  "confidence": "High|Medium|Low",
  "scoring": [
    {
      "option": "Option name",
      "criterion_scores": [
        {"criterion": "Name", "score": 7, "weighted_score": 2.1, "reasoning": "..."}
      ],
      "total_score": 8.4
    }
  ],
  "summary": "One paragraph explaining the trade-offs."
}

Be conservative. If data is missing, lower the score and explain the uncertainty."""

Step 3: Build the evaluator function

Next I write the function that serializes the inputs into a user message and calls Llama 3.3 70B on Oxlo.ai. I set the temperature low and ask for JSON object mode so the output is deterministic and parseable.

def evaluate_decision(
    question: str,
    criteria: List[Criterion],
    options: List[Option],
    model: str = "llama-3.3-70b"
) -> dict:
    criteria_json = json.dumps([c.__dict__ for c in criteria], indent=2)
    options_json = json.dumps([o.__dict__ for o in options], indent=2)

    user_message = f"""Decision question: {question}

Criteria (with weights):
{criteria_json}

Options:
{options_json}

Produce the decision report now."""

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
        temperature=0.2,
    )

    return json.loads(response.choices[0].message.content)

Step 4: Add a critique loop

A single pass can miss hidden risks. I add a second function that feeds the report back into Qwen 3 32B and asks it to act as a skeptical peer reviewer. Because Oxlo.ai uses request-based pricing, this extra pass costs the same flat rate no matter how long the prompt is. You can see the exact rates at https://oxlo.ai/pricing.

def critique_recommendation(
    decision_report: dict,
    question: str,
    model: str = "qwen-3-32b"
) -> dict:
    prompt = f"""You previously produced this decision report for the question: {question}

{json.dumps(decision_report, indent=2)}

Now, critique your own recommendation. Identify:
1. Any criterion where the scoring seems too generous or too harsh.
2. Risks or downsides of the top option that were under-weighted.
3. Scenarios where the second-ranked option would actually be better.

Return strictly JSON:
{{
  "critique": "Paragraph of concerns",
  "revised_confidence": "High|Medium|Low",
  "runner_up_rationale": "When to pick the runner-up instead"
}}"""

    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "You are a skeptical peer reviewer. Be concise."},
            {"role": "user", "content": prompt},
        ],
        response_format={"type": "json_object"},
        temperature=0.3,
    )
    return json.loads(response.choices[0].message.content)

Step 5: Wire it into a runnable script

Finally, I add a main block with a realistic scenario: picking a primary database for a new analytics platform. This gives us something we can run immediately.

if __name__ == "__main__":
    criteria = [
        Criterion("Operational cost", 0.25, "Monthly infra and licensing spend"),
        Criterion("Team familiarity", 0.30, "How well the team knows the tech"),
        Criterion("Scalability", 0.25, "Ability to handle 10x load growth"),
        Criterion("Vendor lock-in risk", 0.20, "Ease of migration if we need to move"),
    ]

    options = [
        Option("PostgreSQL on RDS", "Managed relational database with JSON support"),
        Option("MongoDB Atlas", "Managed document store with horizontal scaling"),
        Option("DynamoDB", "Fully managed NoSQL with on-demand pricing"),
    ]

    question = "Which database should we use for the new user analytics platform?"

    report = evaluate_decision(question=question, criteria=criteria, options=options)
    print("=== DECISION REPORT ===")
    print(json.dumps(report, indent=2))

    critique = critique_recommendation(decision_report=report, question=question)
    print("\n=== CRITIQUE ===")
    print(json.dumps(critique, indent=2))

Run it

Save the file as decision_agent.py, export your key, and execute:

export OXLO_API_KEY="sk-oxlo.ai-..."
python decision_agent.py

When I ran this against the database scenario, Llama 3.3 70B returned a report that scored PostgreSQL on RDS highest because of team familiarity, but the critique pass correctly flagged that the scalability score might be too generous for a pure relational workload. Here is a condensed version of the output:

=== DECISION REPORT ===
{
  "recommendation": "PostgreSQL on RDS",
  "confidence": "Medium",
  "scoring": [
    {
      "option": "PostgreSQL on RDS",
      "criterion_scores": [
        {"criterion": "Operational cost", "score": 7, "weighted_score": 1.75, "reasoning": "Predictable pricing, but storage costs scale."},
        {"criterion": "Team familiarity", "score": 9, "weighted_score": 2.7, "reasoning": "Team has 4 years of production experience."},
        {"criterion": "Scalability", "score": 6, "weighted_score": 1.5, "reasoning": "Vertical scaling only without read replicas."},
        {"criterion": "Vendor lock-in risk", "score": 8, "weighted_score": 1.6, "reasoning": "Standard SQL makes migration straightforward."}
      ],
      "total_score": 7.55
    }
  ],
  "summary": "PostgreSQL on RDS wins on team familiarity and low lock-in, but scalability is a known limitation."
}

=== CRITIQUE ===
{
  "critique": "The scalability score of 6 for PostgreSQL may be optimistic if the analytics platform requires heavy write throughput. DynamoDB's on-demand pricing also reduces operational cost uncertainty for spiky workloads.",
  "revised_confidence": "Medium",
  "runner_up_rationale": "Choose DynamoDB if write volume exceeds 10K events per second or if ops overhead must stay near zero."
}

Wrap-up and next steps

This agent gives you a repeatable way to document why a decision was made. Two concrete ways to extend it:

1. Persist every report to a Git repository or Notion database to build an organizational decision log. Over time you will have a searchable archive of reasoning that outlives any single meeting.

2. For complex architectural decisions that require deeper reasoning, swap Llama 3.3 70B for Kimi K2.6 or DeepSeek V3.2 on Oxlo.ai. Both models handle long context windows and chain-of-thought reasoning well, and because Oxlo.ai charges per request rather than per token, you can send long system prompts and multi-turn critique loops without watching metered costs climb.