DEV Community

shashank ms
shashank ms

Posted on

Using LLMs for Social Science Research

We are going to build a qualitative coding assistant that reads interview transcripts and returns structured themes with supporting quotes. It helps social scientists turn hours of manual thematic analysis into a reproducible pipeline that runs in minutes. I will use Oxlo.ai throughout because its flat per-request pricing removes the cost penalty for sending long transcript chunks.

What you'll need

  • Python 3.10 or newer.
  • The OpenAI SDK. Install it with pip install openai.
  • An Oxlo.ai API key from https://portal.oxlo.ai. The free tier includes 60 requests per day, which is enough to prototype this pipeline.

Step 1: Configure the Oxlo.ai client

I start by importing the SDK and pointing it at Oxlo.ai's OpenAI-compatible endpoint. This single client handles every request in the pipeline.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Prepare a sample interview transcript

For this walkthrough I use a short excerpt from a fictional study on remote work fatigue. In production you would load a file, but a string keeps the example self-contained.

TRANSCRIPT = """
Interviewer: How has working from home affected your daily routine?

Participant A: Honestly, the boundaries disappeared. I used to commute, and that was my buffer between work and home. Now I wake up and I'm already at the office. I find myself checking emails at 10 PM because my laptop is right there.

Interviewer: Do you feel pressure to be online more?

Participant A: Absolutely. There's this unspoken expectation that because you're home, you're always available. My manager sends Slack messages at odd hours, and even though I don't have to answer, I feel anxious if I don't.

Interviewer: Have you tried to establish any boundaries?

Participant A: I tried shutting down my computer at six, but then I just open it again after dinner. It's not sustainable. I think I'm more exhausted now than when I was in the office five days a week.
"""

Step 3: Write the system prompt for qualitative coding

The system prompt locks the model into the role of a disciplined qualitative researcher and forces clean JSON output. I keep it generic so each pipeline step can request its own schema in the user message.

SYSTEM_PROMPT = (
    "You are a qualitative research assistant specializing in thematic analysis. "
    "Follow the user's instructions exactly. "
    "Always return valid JSON with no markdown formatting, no preamble, and no explanation. "
    "Use only the JSON schema requested by the user."
)

Step 4: Chunk the transcript and generate initial codes

Long interviews can dilute the model's attention, so I split the text into chunks and extract raw codes from each one. Because Oxlo.ai charges per request rather than per token, I can send generous chunks without worrying about input length.

import json

def chunk_text(text, max_chars=2000):
    paragraphs = text.strip().split("\n\n")
    chunks = []
    current = ""
    for p in paragraphs:
        if len(current) + len(p) > max_chars:
            chunks.append(current.strip())
            current = p
        else:
            current += "\n\n" + p if current else p
    if current:
        chunks.append(current.strip())
    return chunks

def extract_codes(chunk):
    user_msg = (
        "Perform open coding on the following interview excerpt. "
        "Return a JSON object with a single key 'codes' containing a list of strings. "
        "Each string should be a concise code label capturing one idea in the text.\n\n"
        f"Excerpt:\n{chunk}"
    )
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_msg},
        ],
    )
    raw = response.choices[0].message.content
    raw = raw.replace("

```json", "").replace("```

", "").strip()
    return json.loads(raw).get("codes", [])

chunks = chunk_text(TRANSCRIPT)
all_codes = []
for c in chunks:
    all_codes.extend(extract_codes(c))
print("Extracted codes:", all_codes)

Step 5: Group codes into themes

Next I feed the accumulated codes back to the model and ask it to group them into broader themes with definitions. This mimics the axial coding stage of thematic analysis.

def synthesize_themes(codes):
    user_msg = (
        "You are given a list of open codes extracted from interview transcripts. "
        "Group them into 3 to 5 broader themes. "
        "Return JSON with a single key 'themes'. Each theme must have "
        "'theme_name', 'definition', and 'codes' (the codes belonging to it).\n\n"
        f"Codes: {json.dumps(codes)}"
    )
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_msg},
        ],
    )
    raw = response.choices[0].message.content
    raw = raw.replace("

```json", "").replace("```

", "").strip()
    return json.loads(raw).get("themes", [])

themes = synthesize_themes(all_codes)
print("Synthesized themes:")
for t in themes:
    print(f"- {t['theme_name']}: {t['definition']}")

Step 6: Map evidence quotes to each theme

Finally I send the original transcript and the theme list to the model, asking it to pull verbatim quotes that support each theme. This produces the evidence base that researchers need for credibility checks.

def finalize_report(transcript, themes):
    user_msg = (
        "You are given an interview transcript and a list of themes. "
        "For each theme, find 1 to 3 verbatim quotes from the transcript that best illustrate it. "
        "Return JSON with a single key 'report'. Each item must include "
        "'theme_name', 'definition', and 'quotes' (a list of exact strings from the transcript).\n\n"
        f"Themes: {json.dumps(themes)}\n\n"
        f"Transcript:\n{transcript}"
    )
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_msg},
        ],
    )
    raw = response.choices[0].message.content
    raw = raw.replace("

```json", "").replace("```

", "").strip()
    return json.loads(raw).get("report", [])

report = finalize_report(TRANSCRIPT, themes)

Run it

Putting the pieces together produces a single structured report. Here is the full execution block and an example of what it prints.

if __name__ == "__main__":
    chunks = chunk_text(TRANSCRIPT)
    all_codes = []
    for c in chunks:
        all_codes.extend(extract_codes(c))
    themes = synthesize_themes(all_codes)
    report = finalize_report(TRANSCRIPT, themes)
    print(json.dumps(report, indent=2))

Example output:

[
  {
    "theme_name": "Blurred Work-Home Boundaries",
    "definition": "The collapse of spatial and temporal separation between professional and personal life.",
    "quotes": [
      "the boundaries disappeared. I used to commute, and that was my buffer between work and home.",
      "Now I wake up and I'm already at the office."
    ]
  },
  {
    "theme_name": "Digital Presenteeism",
    "definition": "Pressure to remain constantly available through digital communication tools.",
    "quotes": [
      "There's this unspoken expectation that because you're home, you're always available.",
      "My manager sends Slack messages at odd hours, and even though I don't have to answer, I feel anxious if I don't."
    ]
  },
  {
    "theme_name": "Failed Boundary Restoration",
    "definition": "Unsuccessful attempts to re-establish limits on work activity during off-hours.",
    "quotes": [
      "I tried shutting down my computer at six, but then I just open it again after dinner.",
      "It's not sustainable."
    ]
  }
]

Wrap-up

To scale this beyond a single transcript, wrap the pipeline in a loop that writes results to a JSONL file or a SQLite database. Oxlo.ai's flat per-request pricing makes that cost predictable even when individual interviews run long. Another solid next step is to add an inter-rater reliability check by running the same transcript through two different Oxlo.ai models, such as llama-3.3-70b and qwen-3-32b, then comparing the overlap in generated codes with Cohen's kappa.

Top comments (0)