Building HelpPilot: An IT Helpdesk Autopilot That Knows When to Ask for Help

Mr. Asliddin — Wed, 08 Jul 2026 11:46:54 +0000

Published: true
description: "How I built a multi-agent AI system on Qwen Cloud and Alibaba Cloud — and why the hardest part wasn't the AI at all."

tags: ai, showdev, hackathon, opensource

How I built a multi-agent AI system on Qwen Cloud and Alibaba Cloud — and why the hardest part wasn't the AI at all.

The Idea

Every company with more than a handful of employees has the same quiet problem: an IT helpdesk queue full of noise. Password resets sit next to critical AD replication failures. Five people report the same VPN outage as five separate tickets. A hardware failure with an actual fire risk waits in line behind someone who can't find the printer.

Humans are good at judgment. They're bad at triage at scale. So I built HelpPilot — an autopilot that reads every ticket the moment it lands, understands both the technical problem and the human behind it, and decides: resolve it now, or hand it to a person.

This is the story of building it for the Qwen Cloud Global AI Hackathon, Track 4: Autopilot Agent.

The Architecture: Four Agents, One Pipeline

Instead of one giant prompt trying to do everything, HelpPilot splits the work across four specialized agents, each with a single job:

ClassifierAgent — reads the raw ticket and assigns a category and priority
EmotionAnalyzerAgent — reads the tone — is this person calm, stressed, or about to churn?
KBSearcherAgent — searches a ChromaDB-backed knowledge base for relevant fixes
ResolverAgent — drafts an actual resolution, with a confidence score

Every single step emits a structured event onto an internal event bus, which powers a live "reasoning trace" in the dashboard — so nothing the AI does is a black box. Admins can watch, in real time, exactly what each agent saw and decided.

The Feature I'm Proudest Of: The Trust Dial

Most "autopilot" products are a single switch: automation on, or automation off. That's the wrong shape for how real teams actually think about risk.

The Trust Dial: A per-category autonomy threshold. It turns "how much do I trust the AI" from a binary decision into something closer to how you'd manage a junior team member — different tasks, different levels of oversight, earned over time.

Password resets? Let it auto-resolve instantly, no human needed.
Hardware failures? Never auto-resolve, always escalate.
Billing disputes? Somewhere in between.

Paired with this is a Verified Auto-Remediation Loop: before HelpPilot acts autonomously, it scores its own confidence. Below the Trust Dial's threshold for that category, it doesn't guess — it routes straight to a human, automatically.

The Migration Nobody Warns You About

HelpPilot was originally built against AWS Bedrock. The hackathon required deployment on Qwen Cloud and Alibaba Cloud infrastructure — which meant a full migration of every LLM call to Qwen Cloud's OpenAI-compatible API (https://dashscope-intl.aliyuncs.com/compatible-mode/v1).

The API compatibility made the code migration mechanically simple — swap the client, keep the same request shape. What actually ate the time was everything around the model call: Docker builds failing on native module compilation, a Lightsail instance running out of RAM mid-build, and — memorably — realizing far too late in the process that my auth middleware was rejecting every single request from judges who obviously don't have my private API key.

If there's one lesson from this hackathon, it's that deploying the AI is a completely different skill from building it. The agents themselves came together faster than getting Docker, Nginx, SSL, and DNS to all agree with each other on a fresh Alibaba Cloud ECS instance.

A Feature That Almost Didn't Make It: Voice Control

Late in the build, I added a floating voice button to the dashboard — talk to HelpPilot instead of clicking through it. "How many tickets are pending?" "Approve the first ticket."

The speech-to-text runs entirely in the browser (the Web Speech API — free, no extra latency, no audio upload needed), and the transcript gets sent to a lightweight backend route: fast keyword matching for common commands, falling back to Qwen Cloud for anything free-form. It's a small feature, but it's the one that makes the dashboard feel less like a tool and more like a coworker.

What's Live

HelpPilot is deployed and running at the link below, containerized with Docker on Alibaba Cloud ECS. Every LLM call — classification, emotion analysis, knowledge base search, resolution drafting, and voice command interpretation — routes through Qwen Cloud.

Try HelpPilot Live

The full source is open source under MIT:

ErkinovAsliddin / HelpPilot

HelpPilot — IT Helpdesk Autopilot

HelpPilot is a production-ready Smart IT Helpdesk Autopilot that automatically classifies, prioritizes, searches a knowledge base, and either auto-resolves or drafts responses for human approval.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                       Ingestion Layer                           │
│   REST API (POST /api/tickets)  │  Email (IMAP)  │  Web Form   │
│                  MultiModalHandler (OCR / Whisper / Thread)     │
└──────────────────────────────┬──────────────────────────────────┘
                               │ EventBus: ticket.received
┌──────────────────────────────▼──────────────────────────────────┐
│                    Orchestration Pipeline                       │
│  ClassifierAgent → EmotionAnalyzerAgent → KBSearcherAgent →    │
│  ResolverAgent → [auto_resolve | pending_approval | escalate]  │
└──────────────────────────────┬──────────────────────────────────┘
                               │
┌──────────────────────────────▼──────────────────────────────────┐
│              HITL Approval Layer (Admin Dashboard)              │
│  Approve / Edit-Approve / Reject → DeliveryService             │
└──────────────────────────────┬──────────────────────────────────┘
                               │
┌──────────────────────────────▼──────────────────────────────────┐
│  LoggerAgent → SQLite terminal state + ChromaDB KB feedback     │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│  PredictionEngine (background) → Incident detection + dispatch  │
└─────────────────────────────────────────────────────────────────┘

Quick Start

Prerequisites

Node.js 20+
Docker (for ChromaDB)
AWS credentials with Bedrock access

1. Start ChromaDB

docker run -p 8000:8000 chromadb/chroma

2. Configure

…

View on GitHub

What I'd Do Differently

If I rebuilt this from scratch, I'd containerize and deploy to the target cloud on day one, not as a last step — nearly every late-stage bug I hit (auth misconfiguration, memory limits, missing environment variables) only surfaced once real traffic hit the real infrastructure. Multi-agent AI systems are, ironically, the easy part. Making sure the box they run on doesn't fall over is where the real engineering happens.

Discussion

I'm curious to hear from other DevOps or IT folks: where would you set the "Trust Dial" in your own organization? Would you ever trust an AI to handle billing disputes, or is that strictly human territory?

Built solo for the Qwen Cloud Global AI Hackathon.

DEV Community: Mr. Asliddin

[Boost]