I built a pricing quote bot for an e commerce company. It answered customer questions about product prices.
On day three, a customer asked a simple question. How much is this camera.
The bot replied with prices from three competitors.
The customer asked why the bot was advertising other stores. The bot replied that it was providing a pricing comparison.
The customer never asked for a comparison. They just wanted our price.
The Setup
This was an electronics retailer. Customers asked price questions through chat like how much is this laptop, what is the price of these headphones, or is this item on sale.
The requirement was simple. When someone asks for a price, show the company’s price.
I added web search so the bot could look up current catalog pricing, check sale status, and verify stock. I tested it with forty sample questions. Everything looked perfect.
We deployed on Wednesday.
The Leak Begins
On Thursday morning, the first strange response appeared.
A customer asked for the price of Sony headphones. The bot replied with prices from Amazon, Target, and Walmart, then added our price at the end.
The customer replied asking for our price only.
That was the first red flag.
The Pattern
Every pricing question followed the same pattern.
The customer asked for a price.
The bot searched the web.
The search results contained competitor prices because they ranked highly.
The bot included everything it found.
The response accidentally promoted competitors.
The bot was doing exactly what I told it to do.
Why This Happened
My prompt told the bot to search for current pricing information and be helpful and thorough.
To the model, thorough meant sharing every relevant result it found. Web search does not distinguish between company data and market data. It just returns information.
The bot treated competitor prices as relevant because they appeared in the search results.
This was not a reasoning failure. It was a scope failure.
The Real Problem
The bot could not distinguish between information for the customer and information about the market.
Once web search was added, competitor pricing entered the context. Without strict rules, the model had no reason to hide it.
It assumed that more information was better.
The Failed Fix
I tried telling the bot to only share our prices.
That caused another failure. The bot could not reliably identify which price in the search results belonged to us, so it sometimes refused to answer at all.
The instruction was correct, but the input was still polluted.
The Real Solution
I restricted the search itself.
Instead of searching the entire web, the bot was only allowed to search our own domain. Competitor sites were explicitly excluded.
The rules became simple. Only retrieve pricing from our website. Ignore everything else. Never mention competitors unless the customer explicitly asks for a comparison.
Once competitor data stopped entering the context, the problem disappeared.
The Transformation
The same questions now produced clean answers.
When a customer asked for a product price, the bot returned a single number, stock status, and shipping information.
No competitor names. No comparisons. No accidental promotions.
When a customer explicitly asked whether our price was better than another store, the bot performed a separate comparison workflow and clearly labeled it as such.
Implicit questions stayed company focused. Explicit comparison requests triggered comparison logic.
The Results
Before the fix, competitor prices appeared in most pricing conversations. Customers frequently left the chat after seeing cheaper alternatives. Conversion rates were low and sales teams were frustrated.
After the fix, competitor prices were never mentioned unless requested. Chat abandonment dropped sharply. Conversion rates more than doubled.
The bot stopped acting like a market research tool and started acting like a sales assistant.
What I Learned
Web search without guardrails will always overshare.
Being thorough is dangerous when relevance is not defined.
Customers asking how much want one number, not a price landscape.
Search scope matters more than response wording.
The mistake was not giving the AI too much information. It was letting the wrong information into the context in the first place.
The Bottom Line
The bot quoted competitor prices because I told it to search broadly and be thorough.
The fix was not better phrasing. It was controlling where the bot was allowed to look and what it was allowed to say.
Now the bot answers pricing questions without sending customers to competitors.
Written by FARHAN HABIB FARAZ, Senior Prompt Engineer and Prompt Team Lead at PowerInAI
Building AI that knows what to search and what to share.
Tags: searchintegration, promptengineering, ecommerce, contextmanagement, automation
Top comments (0)