My journey and insights in building a chatbot for eCommerce claims handling

Introduction

In this article, I will share my journey and insights while building a chatbot for automating eCommerce claims handling. The chatbot was designed to handle customer claims, guide users through the claims process, and integrate with existing systems for seamless resolution.

The Problem

In eCommerce, handling customer claims (returns, refunds, damaged goods) is a significant operational burden. Most companies rely on manual processing, which is slow, inconsistent, and expensive to scale. The goal was to automate this process using a conversational AI chatbot that could:

• Understand customer claims in natural language
• Guide customers through the required steps
• Collect necessary information (order details, photos, descriptions)
• Make decisions based on company policies
• Escalate complex cases to human agents

The Plan

The chatbot needed to handle multiple claim types with different workflows. I broke the problem down into:

1. Intent recognition — Understanding what the customer wants
2. Entity extraction — Pulling out order IDs, product names, issue descriptions
3. Dialog management — Managing multi-turn conversations
4. Policy engine — Applying business rules for auto-resolution
5. Handoff — Smooth transition to human agents when needed

Evaluating the Tools

I evaluated several tools and frameworks:

• Amazon Lex — AWS’s conversational AI service. Good integration with AWS ecosystem.
• Dialogflow — Google’s NLU platform. Strong entity recognition.
• Rasa — Open-source, highly customizable. Steeper learning curve.
• Custom LLM-based — Using GPT/Claude for more flexible conversations.

I chose Amazon Lex for its integration with our existing AWS infrastructure and its built-in support for multi-turn conversations.

The Development Stack

• Amazon Lex for conversational AI
• AWS Lambda for fulfillment logic
• DynamoDB for session and claims data
• AWS Lex Web UI (github.com/aws-samples/aws-lex-web-ui) for the frontend
• S3 for storing claim attachments

Operations

Running the chatbot in production taught me several lessons about operations:

• Monitor confidence scores — Low confidence often indicates new intent patterns
• Log everything — Every conversation turn should be logged for training data
• A/B test responses — Small wording changes can dramatically affect resolution rates
• Build escalation paths early — Not every case can be automated

Insights

• Start with the most common claim types first (80/20 rule)
• Users prefer guided flows over open-ended conversation for claims
• Photo upload capability dramatically reduces back-and-forth
• Clear expectations about what the bot can/cannot do reduces frustration
• Regular retraining with real conversation data is essential
• The biggest ROI came from reducing first-response time, not full automation

Building a claims chatbot is as much about understanding business processes as it is about AI technology. The key is starting simple, measuring everything, and iterating based on real user interactions.