DEV Community

shashank ms
shashank ms

Posted on

LLMs in Social Science

#learnai #oxlo #ai

We are building a thematic coding assistant that reads open-ended survey responses and returns structured codes with supporting quotes. It helps social science researchers accelerate qualitative analysis without losing nuance. The entire pipeline runs against Oxlo.ai via the OpenAI SDK.

What you'll need

Step 1: Configure the Oxlo.ai client

I start every project by verifying the connection. The snippet below initializes the client and sends a lightweight request to confirm the key is active.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Say 'Connection OK' and nothing else."},
    ],
)

print(response.choices[0].message.content)

Step 2: Define the coding schema and system prompt

Social science coding works best when the model knows the research question and expected codebook upfront. I treat the system prompt as the codebook. Edit the categories to match your study.

SYSTEM_PROMPT = """You are a qualitative research assistant. Your job is to perform thematic coding on open-ended survey responses.

Research question: What barriers do people face when adopting remote work?

Codebook:
- TECH: Technology or infrastructure barriers
- POLICY: Company policy or management issues
- SOCIAL: Isolation, teamwork, or communication challenges
- WORK-LIFE: Boundary management or burnout
- NONE: No significant barrier mentioned

Instructions:
1. Assign one or more codes to each response.
2. For each code, provide a brief rationale and the exact supporting quote.
3. Return valid JSON only, with no markdown formatting.

JSON schema:
{
  "codes": [
    {"code": "TECH", "rationale": "...", "quote": "..."}
  ]
}"""

Step 3: Build the analysis function

This function wraps the prompt and enforces JSON output. I use Llama 3.3 70B because it follows structured instructions reliably, but Oxlo.ai also offers Qwen 3 32B or Kimi K2.6 if you need multilingual or vision support later.

import json

def code_response(response_text: str) -> dict:
    user_message = f"Survey response:\n{response_text}"
    
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
    )
    
    raw = response.choices[0].message.content
    raw = raw.strip().removeprefix("

```json").removeprefix("```

").removesuffix("

```

").strip()
    return json.loads(raw)

Step 4: Batch process responses

Real fieldwork never involves one response at a time. I loop over a list, sleep briefly between calls to keep the pipeline polite, and collect everything in a list.

import time

survey_data = [
    "My internet is too slow for video calls and the company won't reimburse upgrades.",
    "I miss spontaneous conversations with colleagues. Slack threads feel sterile.",
    "My manager expects immediate replies at 10 PM. There is no boundary.",
    "I love remote work. No barriers at all.",
    "The VPN drops constantly and I lose access to shared drives.",
]

results = []
for idx, text in enumerate(survey_data, 1):
    print(f"Coding response {idx}/{len(survey_data)}...")
    coded = code_response(text)
    results.append({
        "response_id": idx,
        "raw_text": text,
        "codes": coded.get("codes", [])
    })
    time.sleep(0.5)

print(f"Finished coding {len(results)} responses.")

Step 5: Aggregate and export

Now I flatten the results into a frequency table and write both detailed records and a summary to disk. This gives you an audit trail and a quick headline for your methods section.

from collections import Counter

all_codes = []
for r in results:
    for c in r["codes"]:
        all_codes.append(c["code"])

frequency = Counter(all_codes)
print("Code frequency:")
for code, count in frequency.most_common():
    print(f"  {code}: {count}")

with open("coded_survey.json", "w") as f:
    json.dump(results, f, indent=2)

with open("code_frequency.json", "w") as f:
    json.dump(dict(frequency.most_common()), f, indent=2)

print("Files saved: coded_survey.json, code_frequency.json")

Run it

Here is the complete entry point. When I ran this against Oxlo.ai, the output looked like the example below.

if __name__ == "__main__":
    for r in results[:3]:
        print(f"Response {r['response_id']}: {r['raw_text'][:50]}...")
        for c in r["codes"]:
            print(f"  -> {c['code']}: {c['rationale']}")
        print()

Example output:

Response 1: My internet is too slow for video calls and the...
  -> TECH: Respondent identifies inadequate internet speed as a direct obstacle to remote work functionality.
  -> POLICY: Company reimbursement policy is cited as a contributing factor.

Response 2: I miss spontaneous conversations with colleagues. Sla...
  -> SOCIAL: Loss of informal interaction is framed as a teamwork and communication deficit.

Response 3: My manager expects immediate replies at 10 PM. There ...
  -> POLICY: Management expectation is described as an organizational barrier.
  -> WORK-LIFE: Boundary erosion is explicitly mentioned.

Wrap up

Two concrete directions to take this next. First, wire the script into a lightweight Streamlit UI so research assistants can review and override codes before locking the codebook. Second, swap in Oxlo.ai's DeepSeek R1 671B when you move from exploratory coding to the final audit pass. Its chain-of-thought reasoning surfaces edge cases you might miss with a single-pass model.

Top comments (0)