Building a Reading Comprehension Tool with LLM

#learnai #oxlo #ai

I built a reading comprehension tool that ingests arbitrary text passages and answers questions with explicit citations to source sentences. It is useful for students, researchers, and legal teams who need to verify claims against source material without reading every page manually. Because the tool runs on Oxlo.ai, long passages do not inflate costs, since pricing is per request rather than per token.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Initialize the Oxlo.ai client

I start by importing the OpenAI SDK and pointing it at Oxlo.ai. The client is a drop-in replacement, so the only change is the base URL.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Lock down the system prompt

The system prompt keeps the model grounded in the text and prevents hallucination. I keep it in a top-level constant so I can tweak behavior without touching the rest of the code.

SYSTEM_PROMPT = """You are a reading comprehension assistant. Answer questions using only the provided passage.

Rules:
1. Cite the exact sentence or clause that supports your answer.
2. If the passage does not contain the answer, say "The passage does not specify."
3. Keep responses concise but complete. Do not add outside knowledge."""

Step 3: Format passage and question

I need a helper that wraps the passage and question with clear delimiters. This prevents the model from confusing instructions with content.

def build_message(passage: str, question: str) -> str:
    return f"""Passage:
\"\"\"{passage}\"\"\"

Question:
{question}

Answer using only the passage above and cite your source sentence."""

Step 4: Send the request to Oxlo.ai

Now I wire the message into the chat completions endpoint. I use Llama 3.3 70B because it follows instructions tightly and handles long context well. Since Oxlo.ai charges per request, sending a full chapter in the prompt does not change the price.

def answer_question(passage: str, question: str) -> str:
    user_message = build_message(passage, question)

    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )

    return response.choices[0].message.content

Step 5: Add multi-turn memory for follow-ups

Users rarely stop at one question. I extend the script to keep a message history so follow-ups can reference earlier answers while remaining anchored to the original passage.

def chat_about_passage(passage: str, questions: list[str]) -> list[str]:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Reference passage:\n\"\"\"{passage}\"\"\""}
    ]
    answers = []

    for question in questions:
        messages.append({"role": "user", "content": question})

        response = client.chat.completions.create(
            model="llama-3.3-70b",
            messages=messages,
        )

        answer = response.choices[0].message.content
        answers.append(answer)
        messages.append({"role": "assistant", "content": answer})

    return answers

Run it

The script below feeds a paragraph about photosynthesis and asks two questions, including a follow-up that relies on context from the first answer.

PASSAGE = (
    "Photosynthesis is the process by which green plants and some other organisms use sunlight "
    "to synthesize foods with the help of chlorophyll pigments. During photosynthesis in green "
    "plants, light energy is captured and used to convert water, carbon dioxide, and minerals "
    "into oxygen and energy-rich organic compounds. The process occurs primarily in the leaves, "
    "specifically within the chloroplasts. Chlorophyll absorbs light most strongly in the blue "
    "and red portions of the electromagnetic spectrum. A molecule of chlorophyll absorbs a photon, "
    "and this energy is transferred to an electron, initiating the chain of reactions that produce "
    "glucose and oxygen."
)

if __name__ == "__main__":
    print("Single-turn:")
    print(answer_question(PASSAGE, "What pigments are involved?"))
    print()

    print("Multi-turn:")
    for idx, ans in enumerate(chat_about_passage(PASSAGE, [
        "What pigments are involved?",
        "Where does this process primarily occur?"
    ]), 1):
        print(f"{idx}. {ans}")
        print()

Example output:

Single-turn:
Chlorophyll pigments are involved. The passage states that organisms use "sunlight to synthesize foods with the help of chlorophyll pigments."

Multi-turn:
1. Chlorophyll pigments are involved. The passage notes that green plants and other organisms use "sunlight to synthesize foods with the help of chlorophyll pigments."
2. The process primarily occurs in the leaves, specifically within the chloroplasts. This is directly stated in the passage.

Next steps

Add a simple chunking strategy and semantic search layer so the tool can pull relevant paragraphs from large documents instead of loading the entire text into the prompt. For production use, log each request ID and response to a local SQLite database so you can audit citations later.

DEV Community