DEV Community

shashank ms
shashank ms

Posted on

Semantic Role Labeling with LLM: A Step-by-Step Guide

Semantic role labeling extracts who did what to whom from raw text. In this guide, I will build a small Python utility that uses an LLM to turn unstructured sentences into structured predicate-argument frames. It is useful for researchers or engineers who need quick event annotations without training custom models.

What you'll need

Oxlo.ai uses flat per-request pricing, so labeling a batch of long documents costs the same as short ones. That makes it a strong fit for SRL workloads where context windows can grow.

Step 1: Configure the Oxlo.ai client

First, I verify that the client can reach Oxlo.ai and return a response. I will use Llama 3.3 70B, a solid general-purpose model for this task.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "user", "content": "Confirm you are ready to perform semantic role labeling."},
    ],
)

print(response.choices[0].message.content)

If you see a response, the connection is live.

Step 2: Design the SRL system prompt

The system prompt is the most important part of this build. It forces the model to return PropBank-style frames as JSON. I keep the instructions strict and explicit so the output is predictable.

SYSTEM_PROMPT = """You are a semantic role labeling engine.
Given an input sentence, identify each predicate (verb or verb phrase) and label its arguments using PropBank-style roles.
Return a JSON object with this exact structure:
{
  "frames": [
    {
      "predicate": "string",
      "arguments": [
        {"role": "ARG0", "text": "string"},
        {"role": "ARG1", "text": "string"},
        {"role": "ARGM-TMP", "text": "string"}
      ]
    }
  ]
}
Rules:
- ARG0 is typically the agent or causer.
- ARG1 is typically the patient or theme.
- ARGM-* tags cover modifiers such as time, location, or manner.
- Only include arguments that are explicitly present in the sentence.
- If no predicate is found, return an empty frames list."""

Step 3: Build the extractor function

Now I wrap the API call in a reusable function. I use Qwen 3 32B here because it handles structured instruction following well. I also enable JSON mode so the model is constrained to valid JSON.

import json
from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def extract_srl(sentence: str) -> dict:
    response = client.chat.completions.create(
        model="qwen-3-32b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": sentence},
        ],
        response_format={"type": "json_object"},
    )
    
    raw = response.choices[0].message.content
    return json.loads(raw)

# Quick test
result = extract_srl("The engineer submitted the patch before midnight.")
print(json.dumps(result, indent=2))

Step 4: Process multiple sentences

In practice, you will label more than one sentence. Because Oxlo.ai uses per-request pricing instead of per-token pricing, the cost of this batch is predictable and does not scale with sentence length.

sentences = [
    "The engineer submitted the patch before midnight.",
    "Maria gave the visitor a tour of the lab.",
    "The server crashed during the deployment yesterday.",
]

def batch_extract(sentences: list[str]) -> list[dict]:
    results = []
    for sentence in sentences:
        try:
            frame = extract_srl(sentence)
            results.append(frame)
        except Exception as e:
            results.append({"error": str(e), "sentence": sentence})
    return results

batch_results = batch_extract(sentences)
for idx, res in enumerate(batch_results):
    print(f"--- Sentence {idx + 1} ---")
    print(json.dumps(res, indent=2))

Step 5: Add output validation

LLMs can occasionally hallucinate keys or return malformed JSON. I add a thin validation layer to guarantee that every result contains the expected frames list and that each frame has a predicate and arguments.

from typing import Optional

def extract_srl_safe(sentence: str) -> Optional[dict]:
    try:
        parsed = extract_srl(sentence)
        if "frames" not in parsed:
            return {"frames": []}
        for frame in parsed["frames"]:
            if "predicate" not in frame or "arguments" not in frame:
                return {"frames": []}
        return parsed
    except (json.JSONDecodeError, KeyError) as e:
        print(f"Validation failed for: {sentence} ({e})")
        return None

Run it

Here is the complete script entry point. I run the validator over three example sentences and print the structured frames.

if __name__ == "__main__":
    test_sentences = [
        "The engineer submitted the patch before midnight.",
        "Maria gave the visitor a tour of the lab.",
        "The server crashed during the deployment yesterday.",
    ]

    for sent in test_sentences:
        print(f"\nInput: {sent}")
        out = extract_srl_safe(sent)
        print(json.dumps(out, indent=2))

Example output:

Input: The engineer submitted the patch before midnight.
{
  "frames": [
    {
      "predicate": "submitted",
      "arguments": [
        {"role": "ARG0", "text": "The engineer"},
        {"role": "ARG1", "text": "the patch"},
        {"role": "ARGM-TMP", "text": "before midnight"}
      ]
    }
  ]
}

Input: Maria gave the visitor a tour of the lab.
{
  "frames": [
    {
      "predicate": "gave",
      "arguments": [
        {"role": "ARG0", "text": "Maria"},
        {"role": "ARG2", "text": "the visitor"},
        {"role": "ARG1", "text": "a tour of the lab"}
      ]
    }
  ]
}

Input: The server crashed during the deployment yesterday.
{
  "frames": [
    {
      "predicate": "crashed",
      "arguments": [
        {"role": "ARG1", "text": "The server"},
        {"role": "ARGM-TMP", "text": "during the deployment"},
        {"role": "ARGM-TMP", "text": "yesterday"}
      ]
    }
  ]
}

Wrap-up

You now have a working SRL utility backed by Oxlo.ai. Two concrete next steps: wire this into an information extraction pipeline to populate a knowledge graph, or add coreference resolution so you can link arguments like "The engineer" across multiple sentences to track entity-centric event chains.

For details on flat per-request pricing and available models, see https://oxlo.ai/pricing.

Top comments (0)