Building a Coreference Resolution Tool with LLM: A Step-by-Step Guide

#engineering #oxlo #ai

Coreference resolution maps pronouns and descriptive phrases back to the real-world entities they refer to. A lightweight LLM-based resolver is enough to clean up messy transcripts, legal documents, or chat logs without training a custom spaCy model. In this guide I will wire up a working resolver using Oxlo.ai and the OpenAI SDK, taking advantage of flat request-based pricing that stays predictable even when we feed it long paragraphs.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

Step 1: Configure the Oxlo.ai client

I start by instantiating the OpenAI-compatible client pointing at Oxlo.ai. I use Llama 3.3 70B because it follows structured instructions reliably and handles long documents without cold starts.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

Step 2: Define the system prompt

The system prompt forces the model to return only JSON containing entity clusters. Each cluster lists every mention that refers to the same entity, with the most specific mention chosen as the representative.

SYSTEM_PROMPT = """You are a coreference resolution engine. Given a user text, identify all entity clusters. Each cluster groups mentions that refer to the same real-world entity.

Return strictly JSON in this exact structure:
{
  "clusters": [
    {
      "representative": "the clearest full name or noun phrase",
      "mentions": ["span 1", "span 2", "span 3"]
    }
  ]
}

Rules:
- Include pronouns, proper names, and descriptive noun phrases.
- The representative must be the most specific mention in the cluster.
- Do not wrap the JSON in markdown fences.
- Do not add commentary outside the JSON."""

Step 3: Build the resolution function

This helper wraps the API call, strips any accidental markdown fences from the output, and parses the result into a native Python dictionary. Dropping temperature to 0.1 keeps the output deterministic.

import json

def resolve_coreference(text: str, model: str = "llama-3.3-70b"):
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        temperature=0.1,
    )
    
    raw = response.choices[0].message.content.strip()
    # Guard against models that occasionally return fenced JSON.
    if raw.startswith("

```"):
        raw = raw.split("```

")[1]
        if raw.startswith("json"):
            raw = raw[4:]
    
    return json.loads(raw.strip())

Step 4: Add a mention replacer

To make the resolution visible, I add a small utility that swaps every resolved mention with its representative. It processes longer mentions first so that short pronouns do not clobber substrings inside longer names.

def replace_mentions(text: str, clusters: list):
    """
    Naive exact-match replacement: swaps each mention with the representative.
    Processes longer mentions first to avoid partial overwrites.
    """
    for cluster in clusters:
        rep = cluster["representative"]
        mentions = sorted(cluster["mentions"], key=len, reverse=True)
        for mention in mentions:
            text = text.replace(mention, rep)
    return text

Run it

I will feed the tool a paragraph with multiple people and pronouns, then print the raw clusters and the resolved text. Because Oxlo.ai prices by the request rather than by the token, this call costs the same whether the paragraph is two sentences or two pages. See https://oxlo.ai/pricing for plan details.

if __name__ == "__main__":
    paragraph = (
        "Sunita Rao joined the platform team in January. "
        "The backend lead rewrote the auth service, and she cut latency by half. "
        "Marco Diaz took over code review for Sunita. "
        "He merged the branch after he confirmed all edge cases were covered."
    )
    
    result = resolve_coreference(paragraph, model="llama-3.3-70b")
    print(json.dumps(result, indent=2))
    
    resolved = replace_mentions(paragraph, result["clusters"])
    print("\nResolved text:\n" + resolved)

Example output:

{
  "clusters": [
    {
      "representative": "Sunita Rao",
      "mentions": ["Sunita Rao", "The backend lead", "she", "Sunita"]
    },
    {
      "representative": "Marco Diaz",
      "mentions": ["Marco Diaz", "He", "he"]
    }
  ]
}

Resolved text:
Sunita Rao joined the platform team in January. Sunita Rao rewrote the auth service, and Sunita Rao cut latency by half. Marco Diaz took over code review for Sunita Rao. Marco Diaz merged the branch after Marco Diaz confirmed all edge cases were covered.

Next steps

Wire this resolver into an async pipeline so you can process documents in parallel with Oxlo.ai's request-based pricing. For multilingual documents, swap the model to qwen-3-32b or kimi-k2.6 without changing any client code. If you need guaranteed throughput for large backlogs, the Enterprise tier offers dedicated GPUs and custom queue priority.

DEV Community