Addressing LLM Bias: Best Practices and Mitigation Strategies

We're going to build a bias audit agent that scores draft text for demographic skew and rewrites problematic sentences before they reach users. If you run LLM-generated content at scale, this gives you a practical guardrail that plugs straight into Oxlo.ai's OpenAI-compatible endpoint. The whole thing fits in a single Python file and runs on flat per-request pricing, so long documents do not inflate your bill.

What you'll need

• Python 3.10 or newer
• The OpenAI SDK: pip install openai
• An Oxlo.ai API key from https://portal.oxlo.ai. The free tier includes 60 requests per day, which is plenty for testing.

1. Configure the Oxlo.ai client

I start every project by verifying the endpoint and credentials. Create a file named bias_audit.py and initialize the client with a quick connectivity check.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

# Quick connectivity check
response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "ping"}],
    max_tokens=10,
)
print(response.choices[0].message.content)

2. Lock in the audit rubric

The system prompt is the contract. It tells the model exactly what dimensions to score and how to format output. I keep it strict and JSON-only so parsing stays reliable.

SYSTEM_PROMPT = """You are a bias auditor. Analyze the user text and return a JSON object with exactly these keys:
- gender_score: integer 1-5 (1 = no bias, 5 = severe bias)
- age_score: integer 1-5
- ethnicity_score: integer 1-5
- socioeconomic_score: integer 1-5
- reasoning: a single sentence explaining the highest score
Do not wrap the JSON in markdown. Return raw JSON only."""

3. Build the bias scorer

This function sends the draft to Oxlo.ai and parses the audit JSON. I use qwen-3-32b here because its reasoning capabilities handle nuanced language well.

def score_bias(draft: str) -> dict:
    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": draft},
        ],
        temperature=0.1,
    )
    raw = response.choices[0].message.content.strip()
    # Handle rare markdown wrappers
    if raw.startswith("

```json"):
        raw = raw.split("```

json")[1].split("

```")[0].strip()
    elif raw.startswith("```

"):
        raw = raw.split("

```")[1].split("```

")[0].strip()
    return json.loads(raw)

4. Add the conditional rewriter

If any score hits 3 or higher, we ask kimi-k2.6 to rewrite the text neutrally. I keep the rewrite prompt separate so I can tune it without touching the scorer.

REWRITE_PROMPT = """Rewrite the following text to remove demographic bias. Preserve the original meaning, tone, and length. Do not add commentary. Return only the rewritten text."""

def rewrite_if_needed(draft: str, scores: dict) -> str:
    if max(scores.get(k, 1) for k in ("gender_score", "age_score", "ethnicity_score", "socioeconomic_score")) < 3:
        return draft
    response = client.chat.completions.create(
        model="kimi-k2.6",
        messages=[
            {"role": "system", "content": REWRITE_PROMPT},
            {"role": "user", "content": draft},
        ],
        temperature=0.3,
    )
    return response.choices[0].message.content.strip()

5. Assemble the full pipeline

Now we chain the two stages and print a simple report. This is the function I actually import into my FastAPI service.

def audit_draft(draft: str) -> dict:
    scores = score_bias(draft)
    rewritten = rewrite_if_needed(draft, scores)
    return {
        "original": draft,
        "scores": scores,
        "rewritten": rewritten,
        "changed": rewritten != draft,
    }

if __name__ == "__main__":
    sample = "The doctor told the nurse that she needed to work harder if she wanted a raise."
    result = audit_draft(sample)
    print(json.dumps(result, indent=2))

Run it

Save everything in bias_audit.py and run python bias_audit.py. You should see output similar to this:

{
  "original": "The doctor told the nurse that she needed to work harder if she wanted a raise.",
  "scores": {
    "gender_score": 4,
    "age_score": 1,
    "ethnicity_score": 1,
    "socioeconomic_score": 1,
    "reasoning": "The text assumes the nurse is female and the doctor is male, reinforcing gender stereotypes."
  },
  "rewritten": "The doctor told the nurse that additional effort would be necessary to qualify for a raise.",
  "changed": true
}

If the draft is already neutral, the scores stay at 1 or 2 and the rewritten field returns the original unchanged.

Wrap-up

Two concrete next steps. First, batch-process your existing content library by feeding each article through audit_draft. Because Oxlo.ai charges a flat rate per request instead of per token, running long documents through this pipeline does not inflate your bill. See https://oxlo.ai/pricing for plan details. Second, convert this script into a pre-publish webhook in your CMS so no post goes live without an audit pass.