Logging and tracing LLM calls: what a debuggable AI app actually needs

An AI feature that worked in testing produced a wrong answer for one user in production, and there was no way to reconstruct what actually happened — what prompt was sent, what came back, which model version answered.

Log these, structured, on every call

• The exact prompt sent (or a hash plus the template version, if the prompt contains sensitive data).
• Model name and version, temperature and other parameters used.
• Full raw response, latency, and token counts in and out.
• A request ID that threads through your whole system, so one user's report maps to one trace.

Trace, don't just log

A single user action often triggers several LLM calls (retrieval, generation, a safety check). Group them under one trace ID so you can see the whole chain, not disconnected log lines.

See RAG observability: what to log for the retrieval-specific version of this.

About Pranjul Rathour

Pranjul Rathour

Presenting KrishGyan

Pranjul Rathour — GenAI engineer, Kanpur

Speaking at a MeetKats event

Taking questions during a session

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-07.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

• Email: [e-mail]
• Invite / talk menu: https://pranjulrathour.scult.in/invite
• Portfolio & blog: https://pranjulrathour.scult.in
• LinkedIn: https://www.linkedin.com/in/pranjul-rathour/
• X: https://x.com/PranjulRathourx
• Instagram: https://www.instagram.com/pranjulrathour.in/
• Bluesky: https://bsky.app/profile/pranjulrathour.bsky.social
• GitHub: https://github.com/Pranjulrathour

---

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: [e-mail] · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to