Building a Part-of-Speech Tagging Tool with LLM: A Step-by-Step Guide

#engineering #oxlo #ai

I built a lightweight part-of-speech tagger that routes sentences through an LLM and returns structured annotations. It is useful for quick prototyping in NLP pipelines or for labeling data without maintaining spaCy or NLTK installations. I run it against Oxlo.ai because the flat per-request pricing keeps costs predictable even when I throw long paragraphs at the model, and you can see the exact rates at https://oxlo.ai/pricing.

What you'll need

Before running the tagger, make sure you have Python 3.10 or newer installed. You will also need the OpenAI SDK and an API key from your Oxlo.ai dashboard at https://portal.oxlo.ai.

pip install openai

Step 1: Initialize the Oxlo.ai client

Create a file named pos_tagger.py and set up the client pointing to Oxlo.ai. I am leaving the key as a placeholder, but in production I read it from an environment variable.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY",
)

Step 2: Write the system prompt

The system prompt forces the model to emit only a JSON array. I use the Penn Treebank tagset because it is widely understood, and I explicitly forbid markdown or explanations so parsing stays simple.

SYSTEM_PROMPT = """You are a part-of-speech tagging engine.
Analyze the user sentence and return a JSON array.
Each element must be an object with exactly two keys: "word" and "tag".
Use the Penn Treebank tagset.
Do not add explanations or markdown formatting.
Example input: The quick brown fox jumps.
Example output:
[{"word": "The", "tag": "DT"}, {"word": "quick", "tag": "JJ"}, {"word": "brown", "tag": "JJ"}, {"word": "fox", "tag": "NN"}, {"word": "jumps", "tag": "VBZ"}]
"""

Step 3: Build the tagging function

This function sends the text to llama-3.3-70b hosted on Oxlo.ai and parses the response into native Python objects. I set the temperature low to keep the output deterministic, and I strip any accidental code fences before calling json.loads.

import json

def tag_sentence(text: str):
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": text},
        ],
        temperature=0.1,
    )

    raw = response.choices[0].message.content.strip()

    # Strip markdown fences if the model produced them
    if raw.startswith("

```"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()

    return json.loads(raw)

Step 4: Add a CLI wrapper

Finally, add a small argument parser so we can invoke the tool from the terminal and print the results as a simple TSV table.

import argparse

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="POS tagger via Oxlo.ai")
    parser.add_argument("sentence", help="Sentence to tag")
    args = parser.parse_args()

    tags = tag_sentence(args.sentence)

    for item in tags:
        print(f"{item['word']}\t{item['tag']}")

Run it

Save the complete script and run it against a sample sentence. The model returns one tag per token, including punctuation.

python pos_tagger.py "The developer shipped the feature yesterday."

Example output:

The  DT
developer   NN
shipped VBD
the DT
feature NN
yesterday   NN
.   .

Wrap-up and next steps

If you want to turn this into a pipeline, feed the JSON output into a pandas DataFrame and export a labeled dataset for downstream training. If you are working with multilingual text, swap the model string to qwen-3-32b inside the Oxlo.ai client call, it handles mixed languages well.

DEV Community