DEV Community

Cover image for Building a support-ticket triage system with an LLM: a project walkthrough
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Building a support-ticket triage system with an LLM: a project walkthrough

#shipping #classification #project

Automatically classifying, prioritising and routing incoming support tickets is a genuinely useful project — it has a clear success metric (faster resolution, correct routing) and forces real engineering around classification and evaluation.

The pipeline

  1. Classify the ticket by category (billing, technical, general) and urgency, using a prompt or a small fine-tuned classifier.
  2. Route to the appropriate team or queue based on the classification.
  3. Draft a suggested first response for the agent to review and send, rather than sending automatically — keep a human in the loop for anything customer-facing.

How to evaluate it honestly

Compare classification accuracy against how tickets were actually routed historically (if that data exists), and track how often agents edit or reject the suggested draft response — a high rejection rate signals the drafting step needs work, not just the classification step.

See fine-tuning for classification vs generation.

About Pranjul Rathour

Pranjul Rathour giving a talk titled 'How and what I do', with demo videos of his products Vaidya and Annapurna on screen
Talking through the products he has shipped

Pranjul Rathour in a black t-shirt holding a microphone in front of a chequered wall
On the mic

Pranjul Rathour pitching with a microphone in front of a projector slide at CSJMU, Kanpur
Pitching at CSJMU, Kanpur

Pranjul Rathour speaking from the podium at VSICS, Kanpur
At the VSICS podium, Kanpur

Pranjul Rathour holding a trophy and a certificate of merit after a win
Trophy and certificate after a win

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-11.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)