DEV Community

shashank ms
shashank ms

Posted on

Building a Corporate Training Tool with LLM

#engineering #oxlo #ai

I recently shipped an internal training CLI for onboarding engineers to our security policies. It generates targeted quizzes, explains wrong answers, and logs comprehension scores without any custom model hosting. Here is how I built it on Oxlo.ai in under an hour.

What you'll need

Step 1: Set up the Oxlo.ai client

I start with a thin wrapper around the OpenAI SDK pointed at Oxlo.ai. Because Oxlo.ai is fully OpenAI compatible, this is the only client I need for the entire project.

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"  # from https://portal.oxlo.ai
)

Step 2: Design the training system prompt

The system prompt defines the trainer's personality and constraints. I keep it strict: one question at a time, four options, no answers in the first message.

SYSTEM_PROMPT = """You are a concise corporate training assistant.
Your job is to teach the user a topic, then test them with one multiple-choice question at a time.
Rules:
- Ask one question per turn.
- Provide exactly four answer options labeled A, B, C, D.
- Do not reveal the correct answer until the user has responded.
- If the user is wrong, explain why briefly, then ask a follow-up question on the same concept.
- Track a running score mentally and report it after each answer.
"""

Step 3: Build the quiz generator

I feed the topic into the prompt and ask for a fresh question. I use JSON mode so I can parse the options reliably.

def generate_question(topic):
    user_message = (
        f"Generate one multiple-choice question about {topic}. "
        "Return strictly JSON with keys: question (string), "
        "options (list of 4 strings), correct_index (integer 0-3)."
    )

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
    )

    return json.loads(response.choices[0].message.content)

Step 4: Add an explanation engine

When the user guesses wrong, I want a targeted explanation before the next question. I send the previous context back to the model.

def explain_mistake(topic, question, user_answer, correct_index):
    user_message = (
        f"The user answered '{user_answer}' to the question: {question}\n"
        f"The correct index was {correct_index}. Explain why the user's choice was wrong "
        f"in two sentences, then give a concrete example related to {topic}."
    )

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )

    return response.choices[0].message.content

Step 5: Score and log results

I keep a simple dictionary in memory to track correct and total attempts. This is enough for a first version.

scorecard = {"correct": 0, "total": 0}

def record_result(is_correct):
    scorecard["total"] += 1
    if is_correct:
        scorecard["correct"] += 1
    return f"Score: {scorecard['correct']} / {scorecard['total']}"

Step 6: Wire it into a CLI loop

I tie the pieces together in a loop that reads stdin, generates questions, checks answers, and calls the explanation engine on misses.

def run_training_session(topic, rounds=3):
    print(f"Starting training on: {topic}\n")
    labels = ["A", "B", "C", "D"]

    for i in range(rounds):
        q = generate_question(topic)
        print(f"Q{i+1}: {q['question']}")
        for idx, opt in enumerate(q['options']):
            print(f"  {labels[idx]}. {opt}")

        guess = input("Your answer (A/B/C/D): ").strip().upper()
        guess_idx = labels.index(guess) if guess in labels else -1

        is_correct = guess_idx == q['correct_index']
        if is_correct:
            print("Correct.")
        else:
            print("Incorrect.")
            explanation = explain_mistake(
                topic, q['question'], guess, q['correct_index']
            )
            print(explanation)

        print(record_result(is_correct))
        print()

if __name__ == "__main__":
    run_training_session("SOC 2 Type II access controls")

Run it

Save the script as trainer.py, export your key, and run it.

$ export OXLO_API_KEY="sk-oxlo.ai-..."
$ python trainer.py

Starting training on: SOC 2 Type II access controls

Q1: Which control principle restricts access to systems based on job responsibilities?
  A. Least privilege
  B. Defense in depth
  C. Separation of duties
  D. Dual control
Your answer (A/B/C/D): B
Incorrect.
Defense in depth refers to layered security, not role-based restriction. Least privilege means granting only the access required for a specific job function.
Score: 0 / 1

Q2: Under SOC 2 CC6.1, what must be documented when an employee requests elevated access?
  A. The employee's hiring date
  B. The business justification and approval
  C. The vendor contact list
  D. The office location
Your answer (A/B/C/D): B
Correct.
Score: 1 / 2

Wrap-up

This CLI is enough to deploy for a small team today. Two concrete next steps: pipe the scorecard into a SQLite database or Slack webhook so managers can track completion, and swap in kimi-k2.6 if you need advanced reasoning for technical compliance scenarios.

Because Oxlo.ai uses per-request pricing, adding long system prompts and multi-turn history does not inflate cost the way token-based providers do for interactive training sessions. Check current plans at https://oxlo.ai/pricing.

Top comments (0)