Building SupportX AI Assist: A Multi-Agent IT Support System

Iftikar Alam — Tue, 10 Feb 2026 15:20:34 +0000

🚀 Introduction

Have you ever been stuck waiting for IT support to resolve a simple issue like a password reset or a network configuration problem? What if an AI assistant could instantly classify your problem, search through a comprehensive knowledge base, and provide you with a solution in seconds?

Meet SupportX AI Assist — an intelligent IT support system that leverages multi-agent orchestration, vector search, and large language models to provide instant, accurate solutions to common IT problems. And when it can't solve an issue? It automatically escalates it to human IT support with a detailed ticket.

In this blog post, I'll walk you through how I built this system, the technologies I used, and the architectural decisions that make it work seamlessly.

💡 The Problem

Traditional IT support systems face several challenges:

High Response Times: Users often wait hours or days for simple issues
Repetitive Tickets: Support teams spend time answering the same questions repeatedly
Manual Classification: Issues must be manually categorized and routed
Knowledge Silos: Solutions are scattered across wikis, emails, and documents
Inefficient Escalation: No automated way to escalate unresolved issues

I wanted to build a solution that addresses all these pain points using modern AI technologies.

🏗️ The Solution: Multi-Agent Architecture

SupportX AI Assist uses a multi-agent system where specialized AI agents work together to solve IT issues. Here's how the workflow works:

User Query
    ↓
ClassifierAgent (Categorizes the issue)
    ↓
KnowledgeBaseAgent (Searches Azure AI Search with Gemini embeddings)
    ↓
Solution Found? → Yes → Return to User
    ↓
    No
    ↓
NotificationAgent (Escalate via email)

The Three Agents

Classifier Agent 🎯
- Automatically categorizes IT issues into predefined categories (Network Issue, Hardware Issue, Software Bug, Access Request, Password Reset, Other)
- Uses LLM-based classification for accurate categorization
- Ensures the right knowledge base section is searched
Knowledge Base Agent 🔍
- Performs semantic search using Azure AI Search with vector embeddings
- Uses Google Gemini's gemini-embedding-001 model for embeddings (3072 dimensions)
- Matches user queries with relevant solutions using cosine similarity
- Filters results by category for precision
Notification Agent 📧
- Handles ticket creation when no solution is found
- Sends detailed email notifications to IT support
- Includes user query, ticket ID, and context for quick resolution

🛠️ Tech Stack

Building a system like this requires a carefully chosen tech stack. Here's what powers SupportX:

Frontend

Streamlit: Provides a clean, interactive web interface with minimal code
Custom CSS: Polished UI with modern design elements

AI & LLM

AutoGen (pyautogen): Microsoft's multi-agent orchestration framework
Groq: Ultra-fast LLM inference using Llama models (llama-3.3-70b-versatile)
Google Gemini: High-quality embeddings for semantic search

Vector Search

Azure AI Search: Enterprise-grade vector database with hybrid search capabilities
Supports semantic similarity search with filtering

Notifications

SMTP (Gmail): Email notifications for escalations

🔑 Key Technical Decisions

1. Why Multi-Agent Architecture?

Instead of a monolithic LLM approach, I chose a multi-agent system for several reasons:

Separation of Concerns: Each agent has a specific responsibility
Modularity: Easy to update or replace individual agents
Explainability: Clear understanding of which agent made what decision
Reliability: If one agent fails, others can continue working

2. Why Azure AI Search for Vector Storage?

I evaluated several vector databases (Pinecone, Weaviate, Qdrant) but chose Azure AI Search because:

Hybrid Search: Combines vector search with traditional keyword search
Filtering: Can filter by category before semantic search
Scalability: Enterprise-ready with minimal configuration
Integration: Native Azure ecosystem compatibility

3. Why Gemini Embeddings?

Google's Gemini gemini-embedding-001 model offers:

High Dimensionality: 3072-dimensional embeddings capture more nuance
Quality: Superior semantic understanding compared to older embedding models
Cost-Effective: Generous free tier for development and testing

4. Why Groq for LLM Inference?

Groq provides the fastest LLM inference available:

Speed: ~300 tokens/second (vs ~30-50 for typical APIs)
User Experience: Near-instant responses feel conversational
Cost: Competitive pricing with excellent performance

📊 Implementation Highlights

Creating the Vector Index

The knowledge base setup involves:

Loading IT solutions from data/knowledge_base.json
Generating embeddings for each problem using Gemini
Creating Azure AI Search index with vector search configuration
Uploading documents with embeddings and metadata

# Simplified example from create_and_upload_index.py
for entry in knowledge_base:
    # Generate embedding for the problem
    embedding = genai.embed_content(
        model="models/gemini-embedding-001",
        content=entry["problem"]
    )["embedding"]

    # Upload to Azure AI Search
    document = {
        "id": entry["id"],
        "category": entry["category"],
        "problem": entry["problem"],
        "solution": entry["solution"],
        "embedding": embedding
    }

Agent Orchestration with AutoGen

AutoGen makes multi-agent coordination elegant:

# Create the agents
classifier_agent = AssistantAgent(
    name="ClassifierAgent",
    system_message=classifier_prompt,
    llm_config=llm_config
)

knowledge_base_agent = AssistantAgent(
    name="KnowledgeBaseAgent",
    system_message=knowledge_base_prompt,
    llm_config=llm_config
)

# Register tools
register_function(
    search_knowledge_base,
    caller=knowledge_base_agent,
    executor=user,
    name="search_knowledge_base",
    description="Search for IT solutions in the knowledge base"
)

# Create group chat for orchestration
groupchat = GroupChat(
    agents=[user, classifier_agent, knowledge_base_agent, notification_agent],
    messages=[],
    max_round=10
)

Streamlit UI Design

The interface is designed to be intuitive:

# User describes the issue
user_input = st.text_area("Describe your IT issue", height=150)

# AI processes and provides solution
if st.button("🚀 Resolve Now"):
    with st.spinner("SupportX AI Assist is resolving your issue..."):
        user.initiate_chat(recipient=manager, message=user_input)
        st.success("✅ AI Response:")
        st.markdown(final_response)

# User provides feedback
if st.button("✅ Yes, issue resolved"):
    st.success("🎉 Great! We're glad your issue is resolved.")
elif st.button("❌ No, not helpful"):
    ticket_id = generate_ticket_id()
    # Escalate to IT support
    notification_agent.generate_reply(...)

🎯 Real-World Use Cases

Example 1: Password Reset

User Input:

"I forgot my password and can't log into the system."

System Response:

Classifier categorizes as "Password Reset"
Knowledge Base finds relevant solution:
- Step 1: Click "Forgot Password" on login page
- Step 2: Enter your email address
- Step 3: Check your email for reset link
- Step 4: Follow link and create new password
User confirms issue resolved ✅

Example 2: Network Issue

User Input:

"My laptop can't connect to the company Wi-Fi."

System Response:

Classifier categorizes as "Network Issue"
Knowledge Base provides troubleshooting steps:
- Verify Wi-Fi is enabled
- Check correct network name (CorpWiFi-5G)
- Verify password (check with IT if unsure)
- Restart network adapter
- Contact IT if issue persists
User tries solution but still fails ❌
System generates Ticket ID: TKT-A7B3C9
Email sent to IT support with full context

📈 Performance & Results

Metrics

Average Response Time: < 3 seconds
Classification Accuracy: ~95% (based on manual testing)
Knowledge Base Hit Rate: ~70% (solutions found without escalation)
Escalation Rate: ~30% (issues requiring human support)

User Benefits

✅ Instant Answers: No waiting for support ticket responses
✅ 24/7 Availability: Works outside business hours
✅ Consistent Solutions: Same high-quality answers every time
✅ Automatic Escalation: Seamless handoff to human support when needed

IT Support Benefits

✅ Reduced Ticket Volume: Fewer repetitive tickets
✅ Better Context: Escalated tickets include attempted solutions
✅ Knowledge Management: Centralized, searchable solution database
✅ Efficiency: Support teams focus on complex issues

🧪 Challenges & Lessons Learned

Challenge 1: Embedding Dimension Mismatch

Problem: Initially used wrong embedding dimensions (768 vs 3072)

Solution: Verified Gemini model documentation and recreated index with correct dimensions

Challenge 2: Agent Tool Calling Reliability

Problem: Agents sometimes didn't call the search tool

Solution:

Improved system prompts with clearer instructions
Added explicit tool descriptions
Used register_for_llm and register_for_execution properly

Challenge 3: Email Notification Failures

Problem: Gmail blocked automated emails

Solution: Switched to Gmail App Passwords instead of regular passwords

Challenge 4: Category Filtering Issues

Problem: Some queries returned no results due to strict category matching

Solution: Improved classifier prompt to use exact category names from knowledge base

🔮 Future Enhancements

Here are some ideas I'm considering for future versions:

Conversation History: Allow multi-turn conversations for complex troubleshooting
User Analytics: Track common issues and improve knowledge base
Integration with Ticketing Systems: Direct integration with Jira, ServiceNow, etc.
Voice Interface: Add speech-to-text for hands-free support
Multi-Language Support: Serve global teams in multiple languages
Feedback Loop: Automatically improve knowledge base based on user feedback
Administrative Dashboard: Monitor system performance and agent behavior
Custom Agent Training: Fine-tune agents on organization-specific data

🎓 What I Learned

Building SupportX AI Assist taught me several valuable lessons:

Multi-Agent Systems Are Powerful: Breaking down complex problems into specialized agents makes systems more maintainable and reliable
Embeddings Matter: High-quality embeddings (like Gemini) significantly improve search relevance
User Experience Is Critical: Fast responses (thanks to Groq) make the difference between a tool people use and one they avoid
Prompt Engineering Is An Art: Small changes to agent system messages dramatically affect behavior
Vector Search Isn't Magic: Proper indexing, filtering, and retrieval strategies are essential
AutoGen Simplifies Complexity: The framework handles agent coordination, allowing me to focus on business logic

🚀 Getting Started

Want to build your own version? Here's how:

Clone the repository:

   git clone https://github.com/iftikar0016/multi-agent-it-support
   cd SupportX-AI-Assist

Set up environment:

   python -m venv .venv
   .venv\Scripts\activate
   pip install -r requirements.txt

Configure API keys in .env:
- Azure AI Search credentials
- Google Gemini API key
- Groq API key
- Gmail credentials
Create knowledge base index:

   python create_and_upload_index.py

Run the application:

   streamlit run app.py

🌐 Tech Stack Summary

Component	Technology	Purpose
Frontend	Streamlit	Interactive web UI
Orchestration	AutoGen	Multi-agent coordination
LLM	Groq (Llama 3.3)	Fast inference
Embeddings	Google Gemini	Semantic understanding
Vector DB	Azure AI Search	Knowledge retrieval
Notifications	SMTP (Gmail)	Email escalations
Language	Python 3.8+	Core implementation

📝 Conclusion

SupportX AI Assist demonstrates how modern AI technologies can transform traditional IT support. By combining multi-agent systems, vector search, and large language models, we can create systems that provide instant, accurate support while seamlessly escalating complex issues to human experts.

The multi-agent architecture makes the system modular, maintainable, and explainable — critical factors for enterprise adoption. The use of cutting-edge technologies like Gemini embeddings and Groq inference ensures the best possible user experience.

Whether you're building an IT support system, customer service bot, or any other knowledge-retrieval application, the patterns and technologies demonstrated here can serve as a solid foundation.

🔗 Resources

GitHub Repository: SupportX-AI-Assist
AutoGen Documentation: microsoft.github.io/autogen
Azure AI Search: azure.microsoft.com/ai-search
Google Gemini API: ai.google.dev
Groq: groq.com
Streamlit: streamlit.io

💬 Let's Connect

I'd love to hear your thoughts on this project! Have you built similar multi-agent systems? What challenges did you face? Feel free to reach out or contribute to the project.

Questions or suggestions? Open an issue on GitHub or reach out via iftikaralam0016@gmail.com.

Happy Building! 🚀

DEV Community: Iftikar Alam