Building a Coreference Resolution Tool with LLM

#engineering #oxlo #ai

Coreference resolution is the task of determining when two or more expressions in a text refer to the same real-world entity. I built a small command-line tool that uses an LLM to identify mention clusters and rewrite text with resolved pronouns, which saves time when cleaning up transcripts or prepping documents for downstream NLP pipelines. In this tutorial, I will walk through the exact script I shipped, running on Oxlo.ai's flat per-request API so cost stays predictable even when I feed it long passages.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Initialize the Oxlo.ai client

I start by creating resolve.py and configuring the OpenAI SDK to point to Oxlo.ai. I am using Llama 3.3 70B because it handles long documents cleanly and follows structured instructions well.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

# Model choice: Llama 3.3 70B for reliable instruction following on long inputs.
MODEL = "llama-3.3-70b"

Step 2: Write the system prompt

The model needs explicit instructions to output only JSON and to define what constitutes a coreference chain. I treat the system prompt as a config file so I can tweak it without touching the logic.

SYSTEM_PROMPT = """You are a precise coreference resolution engine.

Given a user-supplied text, perform these tasks:
1. Identify all entity mentions, including pronouns, proper nouns, and nominal phrases.
2. Group mentions into coreference chains where each chain refers to the same real-world entity.
3. Pick the most specific mention in each chain as the canonical representative.
4. Output strictly valid JSON with no markdown formatting.

The JSON schema must be:
{
  "chains": [
    {
      "id": 1,
      "mentions": ["list", "of", "exact", "spans"],
      "canonical": "chosen representative"
    }
  ],
  "resolved_text": "original text with all non-canonical mentions replaced by the canonical representative"
}

Rules:
- Preserve the original text's meaning and sentence structure.
- If a mention is ambiguous, create a separate chain.
- Do not add commentary outside the JSON.
"""

Step 3: Build the resolution function

Now I wire the prompt into a function that sends the raw text to Oxlo.ai and parses the response. I request a low temperature to keep the output deterministic, and I strip any stray markdown code fences just in case.

import json

def resolve_coreferences(text: str) -> dict:
    response = client.chat.completions.create(
        model=MODEL,
        temperature=0.1,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
    )

    raw = response.choices[0].message.content.strip()

    # Defensive cleanup in case the model wraps JSON in markdown fences.
    if raw.startswith("

```"):
        raw = raw.split("```

", 2)[1]
        if raw.startswith("json"):
            raw = raw[4:]
        raw = raw.strip()

    return json.loads(raw)

Step 4: Add a CLI wrapper

To make the tool usable on real files, I add a small argparse interface that reads stdin or a file and prints pretty-printed JSON. This is the exact wrapper I use in my own pipeline.

import argparse
import sys

def main():
    parser = argparse.ArgumentParser(description="Coreference resolution via Oxlo.ai")
    parser.add_argument("file", nargs="?", help="Path to text file (defaults to stdin)")
    args = parser.parse_args()

    if args.file:
        with open(args.file, "r", encoding="utf-8") as f:
            text = f.read()
    else:
        text = sys.stdin.read()

    result = resolve_coreferences(text)
    print(json.dumps(result, indent=2, ensure_ascii=False))

if __name__ == "__main__":
    main()

Run it

I create a sample text file and feed it to the script. Because Oxlo.ai uses flat per-request pricing, I do not need to worry about token cost ballooning when I paste in a long article.

$ echo "Alice went to the store. She bought milk because Alice likes cereal. Then she went home." > sample.txt
$ python resolve.py sample.txt

Example output:

{
  "chains": [
    {
      "id": 1,
      "mentions": ["Alice", "She", "Alice", "she"],
      "canonical": "Alice"
    }
  ],
  "resolved_text": "Alice went to the store. Alice bought milk because Alice likes cereal. Then Alice went home."
}

Wrap-up and next steps

If you plan to process a large backlog, Oxlo.ai's flat per-request pricing keeps the bill predictable even when individual documents run long. Two concrete next steps I would recommend: first, batch process a directory of transcripts by wrapping the script in a shell loop. Second, add an optional --model flag to switch between llama-3.3-70b and kimi-k2.6 for multilingual documents, since Oxlo.ai hosts both under the same endpoint and SDK.

DEV Community