DEV Community

shashank ms
shashank ms

Posted on

Engineering LLM-Powered Education Tools: Lessons Learned

We are building a Socratic tutoring agent that guides students through conceptual questions instead of dumping answers. I shipped a version of this for an internal engineering onboarding tool, and the trick was keeping the model from giving away the solution too early. Here is how I built it.

What you'll need

A free Oxlo.ai account includes 60 requests per day, which is plenty for prototyping. When you move to production, the flat per-request pricing keeps costs predictable even when students paste long code blocks or stack traces into the chat. See https://oxlo.ai/pricing for details.

Step 1: Set up the client

First, I verify that I can reach Oxlo.ai. I use the OpenAI SDK as a drop-in replacement and pick a fast model for the smoke test.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

# Quick connectivity check with a free-tier-friendly model
response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[{"role": "user", "content": "Say hello"}],
)
print(response.choices[0].message.content)

Step 2: Define the system prompt

The system prompt is the entire product. I keep it strict: one question at a time, no full solutions, and a clear marker so my code knows when the student has mastered the concept.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a Socratic tutor for undergraduate computer science.
When a student asks a question, identify the core concept they are struggling with.
Do not give the full answer. Instead, ask a single, focused question that nudges
them toward the insight. If they respond incorrectly, give a tiny hint and ask again.
If they respond correctly, confirm briefly and ask the next sub-question.
Once the student has answered all sub-questions correctly, summarize the concept
and end your message with the tag [SESSION_COMPLETE]. Be concise."""

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Topic: Algorithms\nQuestion: Why is merge sort O(n log n)?"},
    ],
    temperature=0.3,
)
print(response.choices[0].message.content)

Step 3: Build the interactive loop

I maintain a message list so the model has full context across turns. I also cap the loop at ten turns to avoid runaway sessions.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY"),
)

SYSTEM_PROMPT = """You are a Socratic tutor for undergraduate computer science.
When a student asks a question, identify the core concept they are struggling with.
Do not give the full answer. Instead, ask a single, focused question that nudges
them toward the insight. If they respond incorrectly, give a tiny hint and ask again.
If they respond correctly, confirm briefly and ask the next sub-question.
Once the student has answered all sub-questions correctly, summarize the concept
and end your message with the tag [SESSION_COMPLETE]. Be concise."""

def run_session(topic, question):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Topic: {topic}\nQuestion: {question}"},
    ]
    print("Tutor is ready. Type 'exit' to quit.\n")

    for turn in range(10):
        response = client.chat.completions.create(
            model="llama-3.3-70b",
            messages=messages,
            temperature=0.3,
        )
        reply = response.choices[0].message.content
        messages.append({"role": "assistant", "content": reply})
        print(f"Tutor: {reply}")

        if "[SESSION_COMPLETE]" in reply:
            break

        user_input = input("Student: ").strip()
        if user_input.lower() == "exit":
            print("Session ended early.")
            break
        messages.append({"role": "user", "content": user_input})
    else:
        print("Reached max turns.")

if __name__ == "__main__":
    run_session("Algorithms", "Why is merge sort O(n log n) instead of O(n^2)?")

Step 4: Add structured evaluation

After the tutor marks the session complete, I run a second model call with JSON mode to verify that the student actually mastered the concept. I use a smaller model here because the task is narrow.

import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY"),
)

SYSTEM_PROMPT = """You are a Socratic tutor for undergraduate computer science.
When a student asks a question, identify the core concept they are struggling with.
Do not give the full answer. Instead, ask a single, focused question that nudges
them toward the insight. If they respond incorrectly, give a tiny hint and ask again.
If they respond correctly, confirm briefly and ask the next sub-question.
Once the student has answered all sub-questions correctly, summarize the concept
and end your message with the tag [SESSION_COMPLETE]. Be concise."""

def get_tutor_reply(messages):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=messages,
        temperature=0.3,
    )
    return response.choices[0].message.content

def evaluate_mastery(messages):
    eval_prompt = (
        "You are an evaluator. Review the conversation and decide if the student "
        "has demonstrated correct understanding of the concept. "
        'Respond with valid JSON only: {"mastered": true/false}'
    )
    eval_messages = messages + [{"role": "user", "content": eval_prompt}]
    response = client.chat.completions.create(
        model="deepseek-v3.2",
        messages=eval_messages,
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    return json.loads(response.choices[0].message.content)

def run_session(topic, question):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Topic: {topic}\nQuestion: {question}"},
    ]
    print("Tutor is ready. Type 'exit' to quit.\n")

    for turn in range(10):
        reply = get_tutor_reply(messages)
        messages.append({"role": "assistant", "content": reply})
        print(f"Tutor: {reply}")

        if "[SESSION_COMPLETE]" in reply:
            result = evaluate_mastery(messages)
            if result.get("mastered"):
                print("Evaluator: Concept mastered.")
            else:
                print("Evaluator: Session ended but mastery unclear.")
            break

        user_input = input("Student: ").strip()
        if user_input.lower() == "exit":
            print("Session ended early.")
            break
        messages.append({"role": "user", "content": user_input})
    else:
        print("Reached max turns.")

if __name__ == "__main__":
    run_session("Algorithms", "Why is merge sort O(n log n) instead of O(n^2)?")

Run it

Save the script as tutor.py, set your key, and run it.

export OXLO_API_KEY="sk-oxlo.ai-..."
python tutor.py

Here is a real transcript from my test run using llama-3.3-70b on Oxlo.ai:

Tutor is ready. Type 'exit' to quit.

Tutor: Before we jump to the answer, let's look at the structure of merge sort. It recursively splits the array. If you start with n elements, roughly how many split levels are there until every sub-array has one element?
Student: log n
Tutor: Right, log base 2 of n levels. Now think about the merge step. At any single level of the recursion tree, how many total elements are being merged across all sub-arrays?
Student: n
Tutor: Exactly. So you have log n levels, and each level does O(n) work. What is the overall complexity?
Student: n times log n
Tutor: Correct. That is why merge sort is O(n log n). Each level is linear, and there are logarithmic levels. [SESSION_COMPLETE]
Evaluator: Concept mastered.

Wrap up

This pattern works because the system prompt constrains the model tightly. If you are tutoring a more advanced subject, swap llama-3.3-70b for kimi-k2.6 on Oxlo.ai to get deeper reasoning without changing any client code.

Two concrete next steps: wire the loop into a web socket so students can use it in a browser, and store the evaluator's JSON output in your database so you can track actual learning outcomes per student.

Top comments (0)