# Building ZeroxAI: Building ZeroxAI: 10 Days, One Voice Agent, Countless Lessons
Why I Built This
Returns and refunds sound simple until you actually have to deal with one.
A customer may want to return an item, ask why a refund has not arrived, check whether an order is eligible for return, or simply want to know what happens next.
For a local business, these conversations can quickly become repetitive.
The problem is not that businesses don't want to help. It's that the same questions are asked again and again, while customers usually don't want to fill out another form or search through a website just to understand the status of a return.
That is where ZeroxAI comes in.
ZeroxAI is my voice-agent project for the Local Commerce track, built as a Returns & Refunds Specialist.
Instead of making the customer type everything, the idea is simple:
Just talk to the agent.
The agent is designed around one focused responsibility — helping customers with return and refund-related conversations.
I built ZeroxAI as part of the VoiceForBharat journey, where the goal was not just to make an AI speak, but to explore what happens when voice AI is given a real business responsibility.
What ZeroxAI Is Designed to Do
The main idea behind ZeroxAI is specialization.
Rather than building one general-purpose assistant that tries to answer everything, I wanted to create an agent with a much narrower job.
ZeroxAI focuses on conversations such as:
- Return-related questions
- Refund-related questions
- Understanding a customer's issue
- Explaining the next step in a return or refund process
- Asking for the information required to understand the request
- Keeping the conversation natural instead of forcing the customer through a long form
- Knowing when a request should be handled by a human instead
The important part is the last one.
A useful voice agent should not pretend it can solve everything.
Sometimes the best response is not another AI-generated answer.
Sometimes it is:
"I'll connect you with someone who can help."
The Idea Behind the Agent
When I started working on ZeroxAI, I kept coming back to one question:
What would make a voice agent actually useful for a local business?
The answer wasn't simply better voice quality.
It was focus.
A customer calling about a refund doesn't need an AI that can discuss ten different topics.
They need an agent that understands their problem and helps them move forward.
So I treated ZeroxAI more like a specialized customer-support representative than a general chatbot.
The conversation should feel natural, but the agent should still have a clear purpose.
How the Voice Experience Works
The basic experience is straightforward.
A customer speaks through the microphone.
The voice input is processed by the voice-agent pipeline.
The agent understands the request, reasons about what the customer is asking, and produces a response.
That response is then converted back into speech.
The result is a conversation instead of a traditional form-based support flow.
The current VoiceForBharat workspace provides the foundation for microphone input, speech responses, browser speech synthesis, and Murf Falcon TTS integration.
The architecture can be thought of as:
Customer
↓
Voice Input
↓
Speech Recognition
↓
AI Agent
↓
Returns / Refunds Logic
↓
Response
↓
Text-to-Speech
↓
Customer
The interesting engineering work is not just connecting these pieces.
It is deciding what the agent should do with the information moving through them.
The Returns & Refunds Specialist
One of the decisions I made with ZeroxAI was to keep the agent's responsibility narrow.
The specialist should be able to understand questions like:
"Mera order return kab hoga?"
"Refund abhi tak nahi aaya."
"Is product ko return kar sakta hoon?"
"Refund kitne din mein milega?"
The customer shouldn't have to translate their problem into a specific command.
They should be able to explain it naturally.
That is one of the reasons I wanted this project to be voice-first.
Real customers don't always speak in perfectly structured sentences.
They explain problems the same way they would explain them to another person.
Keeping the Conversation Human
One thing I learned while building voice agents is that technically correct responses can still feel terrible.
A chatbot can say:
"Please provide your order ID."
But a voice conversation needs more context.
A better interaction might be:
"Sure, I can help you check that. Could you share your order ID?"
That difference looks small in text.
In a voice conversation, it makes a big difference.
The goal with ZeroxAI is therefore not to make every response long.
It is to make the conversation feel clear, short, and natural.
Safety and Boundaries
Customer-support agents deal with information that can become sensitive very quickly.
A returns and refunds specialist should therefore have clear boundaries.
For example, the agent should not invent a refund status simply because the customer expects one.
It should not promise a refund without the required information.
And if the request requires a human decision, the agent should recognize that instead of pretending to be certain.
This was an important lesson for me:
A good AI agent is defined as much by what it refuses to do as by what it can do.
My VoiceForBharat Journey
I started the project from the VoiceForBharat starter workspace and then focused on the Local Commerce use case.
The repository is structured around the voice-agent workspace with backend, frontend, challenge material, and the Murf/LiveKit starter components.
The journey became less about building a demo and more about thinking through the complete customer interaction.
Day 1 — Getting Voice Working
The first goal was simple:
Get the agent to listen and respond.
I worked with the starter voice-agent setup and established the basic interaction between the microphone and the AI response.
At this stage, the system was intentionally simple.
The important thing was getting the voice loop working.
Day 2 — Giving ZeroxAI an Identity
Once the basic voice interaction worked, the next step was defining what ZeroxAI actually is.
Not a general chatbot.
Not a random voice assistant.
A Returns & Refunds Specialist for Local Commerce.
That distinction became the foundation for the rest of the project.
Day 3 — Designing the Customer Conversation
The next challenge was conversation design.
What should ZeroxAI ask first?
What information does it actually need?
How should it respond when the customer gives incomplete information?
What happens when the customer changes the topic?
These questions made me realize that voice-agent development is partly software engineering and partly conversation design.
Day 4 — Making the Agent More Useful
The next step was moving from simple responses toward useful support interactions.
Instead of thinking:
"What should the AI say?"
I started thinking:
"What should the customer be able to accomplish?"
That shift changed how I approached the agent.
Day 5 — Handling Returns
The specialist's first major responsibility was understanding return-related conversations.
A customer might want to know whether an item can be returned, what information is needed, or what the next step should be.
The agent needs to understand the intent before responding.
Day 6 — Handling Refund Questions
Refund conversations introduce another layer of complexity.
Customers may ask whether a refund has been initiated, when they should expect it, or why it has not appeared yet.
The important part is avoiding confident guesses.
If the system doesn't have enough information, the agent should say so.
Day 7 — Knowing When to Escalate
This became one of the most important parts of the project.
There are situations where the AI should stop trying to solve the problem.
For example, a customer may have a complicated dispute or a situation that requires a business representative.
That's where human escalation becomes part of the design.
The objective is not:
AI handles everything.
The objective is:
AI handles what it can, and knows when to step aside.
Day 8 — Improving the Voice Experience
At this stage, I focused on making the interaction feel less like a technical demo.
Shorter responses.
Clearer questions.
Better conversation flow.
Less unnecessary information.
Voice interfaces punish overly long answers because users have to listen to every word.
Day 9 — Testing Real Conversations
This is where the interesting problems appeared.
A voice agent can work perfectly in a simple demo and still behave strangely when the conversation becomes unpredictable.
So I started testing scenarios such as:
- Customer gives incomplete information
- Customer changes the subject
- Customer repeats the same question
- Customer asks something outside the agent's responsibility
- Customer wants a human
- Customer doesn't provide enough information
- Customer expects an immediate refund confirmation
These tests were more valuable than simply checking whether the agent could speak.
Day 10 — Looking at the Bigger Picture
By the end, I stopped thinking about ZeroxAI as just a voice bot.
It became a small experiment in how AI could support local commerce.
The voice interface is only one part.
The real product is the workflow behind the conversation.
What I Learned
The biggest lesson from building ZeroxAI is that a voice agent isn't just an AI model connected to a microphone.
There are many decisions behind a good interaction:
- What should the agent know?
- What should it ask?
- What information should it remember?
- Which actions should require confirmation?
- What should happen when information is missing?
- When should it stop?
- When should a human take over?
- How do we know whether the customer actually got help?
These questions changed how I think about voice AI.
The Biggest Challenge
The hardest part wasn't making ZeroxAI talk.
The harder part was making it behave consistently.
A text chatbot can hide some awkwardness because users can read, scroll, and reread.
Voice is different.
If the agent asks a confusing question, the customer has to listen to it.
If the agent talks too much, the experience becomes frustrating.
If the agent gives an uncertain answer confidently, trust disappears.
That makes conversation design just as important as the underlying AI.
What I'd Improve Next
ZeroxAI still has plenty of room to grow.
Some of the next things I'd like to explore are:
- Better order and return-status integrations
- More robust refund workflows
- Multilingual and Hinglish conversations
- Better handling of noisy real-world audio
- Customer memory with privacy-aware storage
- More detailed support analytics
- Better human handoff workflows
- Automated follow-up conversations
- More realistic testing with real customer scenarios
- Stronger safeguards around customer and order information
The goal would be to move from a voice-agent experiment toward something that could actually fit into a local commerce support workflow.
What Voice AI Taught Me
When I started this project, I thought the interesting part would be getting an AI to speak.
After building ZeroxAI, I think differently.
Voice is only the beginning.
The real engineering starts when you ask:
What should the agent do next?
A useful agent needs to understand its responsibility.
It needs to ask the right questions.
It needs to avoid making things up.
It needs to know when it has enough information.
And most importantly, it needs to know when a human should take over.
That is what I wanted ZeroxAI to explore.
Not just an AI that talks.
An AI that knows how to help.
Repository
The project is available on GitHub:
ZeroxAI — VoiceForBharat
https://github.com/suhelalase/VoiceForBharat
The repository contains the VoiceForBharat project workspace and starter implementation used for the journey.
Final Thoughts
Building ZeroxAI gave me a different perspective on customer support.
A good voice agent doesn't need to sound impressive every second.
It needs to make the customer's next step easier.
If someone can call, explain a return or refund problem naturally, get a clear answer, and reach a human when necessary, then the technology is actually doing something useful.
That's the direction I want to keep exploring with ZeroxAI.
From voice interaction to useful action.
Top comments (0)