Building Semantic Role Labeling Tools with LLMs

Semantic Role Labeling (SRL) is the task of mapping natural language sentences to predicate-argument structures: identifying the verb or predicate and labeling each argument with its semantic role, such as Agent, Patient, or Instrument. Traditional SRL pipelines depend on syntactic parsers, frame lexicons, and supervised classifiers that are expensive to maintain and brittle across domains. Large language models can collapse this stack into a single, generalizable inference call, extracting structured role frames from raw text with minimal scaffolding.

What Is Semantic Role Labeling

In SRL, a predicate (usually a verb) anchors a frame, and each constituent in the sentence is classified by the role it plays relative to that predicate. For example, in "The engineer deployed the fix with a script," the predicate is "deployed," "The engineer" is the Agent, "the fix" is the Patient, and "with a script" is the Instrument. PropBank and FrameNet define standard role sets, but custom schemas are common in production pipelines for legal, medical, or financial text.

Why Use LLMs for SRL

Classical SRL requires tokenization, part-of-speech tagging, dependency parsing, and then a role classifier. Each component introduces error accumulation and domain drift. LLMs absorb syntax and world knowledge from pretraining, so they can perform zero-shot or few-shot SRL from a prompt and a JSON schema. This removes the need to maintain separate frame semantic resources and retrain models when your domain shifts.

Designing an LLM-Powered SRL Pipeline

A production SRL tool needs deterministic output. The safest approach is to pair a strict system prompt with JSON mode or constrained decoding. Define a schema with fields for the predicate, a list of arguments, and each argument's role and text span. Keep the prompt explicit: instruct the model to respect your custom role ontology, to handle nominal predicates (e.g., "destruction" as a predicate), and to return an empty array when no valid frame is detected.

Code Example: Zero-Shot SRL

The following Python script uses the OpenAI SDK with an Oxlo.ai endpoint. Because Oxlo.ai charges a flat rate per request, sending a long document with detailed instructions and few-shot examples costs the same as a minimal prompt. This is especially useful for SRL, where input paragraphs can grow quickly.

import openai
import json

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

system_prompt = """You are a semantic role labeler.
Extract every predicate-argument structure from the input sentence.
Return JSON with this schema:
{
  "frames": [
    {
      "predicate": "string",
      "arguments": [
        {"text": "string", "role": "string"}
      ]
    }
  ]
}
Use roles: Agent, Patient, Instrument, Beneficiary, Theme, Location.
If a role does not apply, omit it."""

user_input = "The security team patched the vulnerability using an automated harness before the deadline."

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_input}
    ],
    response_format={"type": "json_object"}
)

result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2))

Replace the model identifier with the exact slug from the Oxlo.ai model catalog. Options such as Llama 3.3 70B, Qwen 3 32B, or DeepSeek V3.2 work well for structured extraction tasks. DeepSeek V3.2 is also available on the free tier, so you can prototype without cost.

Improving Accuracy with Few-Shot Prompting

Zero-shot SRL is effective for simple sentences, but ambiguous predicates and nested clauses benefit from examples. Add two or three labeled sentences to the message history before the target input. This guides the model toward your preferred boundary detection and role assignment without any gradient updates. Because Oxlo.ai does not meter input tokens separately, adding these examples does not change your unit cost per request.

Handling Long Documents and Batching

Real-world SRL often runs over full reports, legal filings, or conversation transcripts rather than isolated sentences. Long-context models are essential here. Oxlo.ai hosts DeepSeek V4 Flash, which offers a 1 million token context window, and Kimi K2.6, which supports 131K tokens and advanced reasoning. Running SRL over an entire document in one request preserves cross-sentence coreference and discourse context that sentence-level pipelines lose.

On token-based providers, a long-document SRL job can become prohibitively expensive because input costs scale with every token in the source text. Oxlo.ai uses request-based pricing: one flat cost per API call regardless of prompt length. For long-context and agentic workloads, this can be 10-100x cheaper than token-based billing. See the pricing page for plan details.

Evaluating and Iterating

LLM-based SRL should be evaluated for boundary exactness and role correctness against a gold-standard set. Use JSON mode to eliminate syntax errors, and add a validation layer in Python to ensure returned roles belong to your ontology. If you need multi-turn verification, you can implement a second pass where the model checks its own output for consistency. Oxlo.ai supports multi-turn conversations, streaming responses, and function calling, so you can build an agentic validation loop without switching providers.

Bringing It to Production

Oxlo.ai is fully OpenAI SDK compatible, so deploying an SRL service requires only a base URL change. There are no cold starts on popular models, which means your annotation pipeline responds immediately under load. For sustained throughput, the Pro plan offers 1,000 requests per day across all models, and the Premium plan provides 5,000