Unlocking Semantic Role Labeling with LLMs

#learnai #oxlo #ai

We are going to build a semantic role labeling pipeline that reads raw sentences and extracts predicates with their associated arguments. This kind of structured extraction is useful for anyone building knowledge graphs, event extractors, or advanced search pipelines. I use Oxlo.ai because its flat per-request pricing keeps costs predictable even when we send long documents for analysis, and you can verify current rates at https://oxlo.ai/pricing.

What you'll need

Python 3.10 or newer
The OpenAI SDK: pip install openai
An Oxlo.ai API key from https://portal.oxlo.ai

I also assume you have a virtual environment activated. If you want to compare results across model families, Oxlo.ai hosts Llama 3.3 70B, Qwen 3 32B, and Kimi K2.6 under the same API and pricing structure.

Step 1: Initialize the client

First, I create an OpenAI-compatible client pointed at Oxlo.ai. I test the connection with a lightweight call to make sure my key and base URL are correct.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="deepseek-v3.2",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Say OK"},
    ],
    max_tokens=10,
)
print(response.choices[0].message.content)

Step 2: Define the SRL schema and system prompt

Semantic role labeling works best when the model outputs strict JSON. I define a system prompt that lists the roles we care about and forces a predictable structure.

SYSTEM_PROMPT = """You are a semantic role labeling engine.
For the sentence provided, identify the main predicate and all associated arguments.
Output valid JSON with this exact structure:
{
  "predicate": "string",
  "arguments": [
    {"role": "Agent", "text": "string"},
    {"role": "Patient", "text": "string"},
    {"role": "Theme", "text": "string"},
    {"role": "Instrument", "text": "string"},
    {"role": "Location", "text": "string"},
    {"role": "Time", "text": "string"},
    {"role": "Manner", "text": "string"}
  ]
}
Only include arguments that are actually present in the sentence. Do not hallucinate roles."""

Step 3: Build the extraction function

Now I wire the prompt into a reusable function. I use Llama 3.3 70B for reliable instruction following, and I set the response format to JSON to reduce parsing errors.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a semantic role labeling engine.
For the sentence provided, identify the main predicate and all associated arguments.
Output valid JSON with this exact structure:
{
  "predicate": "string",
  "arguments": [
    {"role": "Agent", "text": "string"},
    {"role": "Patient", "text": "string"},
    {"role": "Theme", "text": "string"},
    {"role": "Instrument", "text": "string"},
    {"role": "Location", "text": "string"},
    {"role": "Time", "text": "string"},
    {"role": "Manner", "text": "string"}
  ]
}
Only include arguments that are actually present in the sentence. Do not hallucinate roles."""

def extract_roles(sentence: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": sentence},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

Step 4: Add batch processing and error handling

Single sentences are useful, but most pipelines process many at once. I add a batch wrapper that catches JSON decoding failures and returns a safe fallback.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a semantic role labeling engine.
For the sentence provided, identify the main predicate and all associated arguments.
Output valid JSON with this exact structure:
{
  "predicate": "string",
  "arguments": [
    {"role": "Agent", "text": "string"},
    {"role": "Patient", "text": "string"},
    {"role": "Theme", "text": "string"},
    {"role": "Instrument", "text": "string"},
    {"role": "Location", "text": "string"},
    {"role": "Time", "text": "string"},
    {"role": "Manner", "text": "string"}
  ]
}
Only include arguments that are actually present in the sentence. Do not hallucinate roles."""

def extract_roles(sentence: str) -> dict:
    try:
        response = client.chat.completions.create(
            model="llama-3.3-70b",
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": sentence},
            ],
            response_format={"type": "json_object"},
            temperature=0.1,
        )
        return json.loads(response.choices[0].message.content)
    except Exception as e:
        return {"error": str(e), "sentence": sentence}

def batch_extract(sentences: list[str]) -> list[dict]:
    return [extract_roles(s) for s in sentences]

Step 5: Validate outputs against the input text

LLMs occasionally invent arguments. I add a lightweight validator that checks every argument text appears as a substring in the original sentence before returning the result.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a semantic role labeling engine.
For the sentence provided, identify the main predicate and all associated arguments.
Output valid JSON with this exact structure:
{
  "predicate": "string",
  "arguments": [
    {"role": "Agent", "text": "string"},
    {"role": "Patient", "text": "string"},
    {"role": "Theme", "text": "string"},
    {"role": "Instrument", "text": "string"},
    {"role": "Location", "text": "string"},
    {"role": "Time", "text": "string"},
    {"role": "Manner", "text": "string"}
  ]
}
Only include arguments that are actually present in the sentence. Do not hallucinate roles."""

def extract_roles(sentence: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": sentence},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    data = json.loads(response.choices[0].message.content)

    text_lower = sentence.lower()
    clean_args = []
    for arg in data.get("arguments", []):
        if arg.get("text", "").lower() in text_lower:
            clean_args.append(arg)
    data["arguments"] = clean_args
    return data

Run it

Here is the complete script with a few test sentences. I run it from the terminal with python srl.py.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

SYSTEM_PROMPT = """You are a semantic role labeling engine.
For the sentence provided, identify the main predicate and all associated arguments.
Output valid JSON with this exact structure:
{
  "predicate": "string",
  "arguments": [
    {"role": "Agent", "text": "string"},
    {"role": "Patient", "text": "string"},
    {"role": "Theme", "text": "string"},
    {"role": "Instrument", "text": "string"},
    {"role": "Location", "text": "string"},
    {"role": "Time", "text": "string"},
    {"role": "Manner", "text": "string"}
  ]
}
Only include arguments that are actually present in the sentence. Do not hallucinate roles."""

def extract_roles(sentence: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": sentence},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )
    data = json.loads(response.choices[0].message.content)

    text_lower = sentence.lower()
    clean_args = []
    for arg in data.get("arguments", []):
        if arg.get("text", "").lower() in text_lower:
            clean_args.append(arg)
    data["arguments"] = clean_args
    return data

if __name__ == "__main__":
    tests = [
        "The chef cooked the soup with a wooden spoon in the kitchen yesterday.",
        "Maria sent a package to her brother in Tokyo last Monday.",
        "The committee approved the new policy by a narrow margin."
    ]
    for t in tests:
        print(json.dumps(extract_roles(t), indent=2))

Example output:

{
  "predicate": "cooked",
  "arguments": [
    {"role": "Agent", "text": "The chef"},
    {"role": "Patient", "text": "the soup"},
    {"role": "Instrument", "text": "a wooden spoon"},
    {"role": "Location", "text": "in the kitchen"},
    {"role": "Time", "text": "yesterday"}
  ]
}
{
  "predicate": "sent",
  "arguments": [
    {"role": "Agent", "text": "Maria"},
    {"role": "Patient", "text": "a package"},
    {"role": "Theme", "text": "to her brother"},
    {"role": "Location", "text": "in Tokyo"},
    {"role": "Time", "text": "last Monday"}
  ]
}
{
  "predicate": "approved",
  "arguments": [
    {"role": "Agent", "text": "The committee"},
    {"role": "Patient", "text": "the new policy"},
    {"role": "Manner", "text": "by a narrow margin"}
  ]
}

Wrap-up

This pipeline gives you a working SRL extractor in under fifty lines of code. The flat per-request pricing on Oxlo.ai means you can throw long paragraphs at the model without watching token meters spin up.

Two concrete next steps: first, swap in qwen-3-32b or kimi-k2.6 to test multilingual or vision-capable SRL on scanned document text. Second, accumulate a few thousand labeled examples and distill them into a smaller fine-tuned classifier for latency-sensitive production paths. You can host that distilled model on Oxlo.ai too.

DEV Community