A grocery assistant can answer “Where is the bread?” in a second.
That is not the interesting part.
The harder problem is answering the next question correctly when the customer has already told the system something important earlier.
For example:
“Naku brown bread ante chala istam, white bread vaddu.”
Later:
“Which bread should I buy?”
A normal chatbot can understand the second sentence. A useful grocery assistant should connect it to the first one.
That is what I built with the GrocerAI Customer Agent: an AI shopping assistant that combines real-time store data, tool-based actions, multilingual conversations, customer-specific memory, and a persistent learning loop.
The goal was not to build another chatbot.
The goal was to make the customer experience become more useful over time.
The Problem With a Normal Shopping Chatbot
A grocery assistant has to answer questions about things that change constantly:
- Is a product available?
- Where is it located?
- What alternatives are available?
- What is already in the cart?
- What is still needed for the shopping mission?
- Has the customer mentioned a preference before?
These are different kinds of information.
Some belong to the current store state.
Some belong to the customer's current session.
Some are experiences that may become useful later.
I therefore designed the Customer Agent around three layers:
Current Store State
+
Current Customer State
+
Relevant Past Experience
↓
Customer Agent
↓
Action / Response
The structured backend remains the source of truth for current facts. Hindsight provides the memory layer for experiences that can influence future reasoning.
The Customer Agent Is Tool-Driven
The LLM is responsible for understanding the customer's request and deciding what information or action is needed.
It does not directly invent inventory or shelf locations.
Instead, it uses backend tools such as:
searchProducts()
getProductDetails()
checkAvailability()
getShelfLocation()
getStoreMapRoute()
getCustomerMission()
updateMission()
getCart()
addToCart()
removeFromCart()
getRecommendations()
checkSubstitution()
sendNotification()
Consider:
“Do you have brown bread?”
The flow can become:
Customer
↓
Customer Agent
↓
searchProducts()
↓
checkAvailability()
↓
getShelfLocation()
↓
Answer
That distinction matters.
The model handles reasoning.
The application handles authoritative store state.
This makes the assistant much more useful than simply asking an LLM to generate an answer from its own knowledge.
Adding Persistent Customer Memory
The biggest change came when I added Hindsight.
The Customer Agent can retain useful experiences and recall them when a future interaction makes them relevant.
Conceptually:
Customer Interaction
↓
retain()
↓
Customer Memory
↓
Future Interaction
↓
recall()
↓
Personalized Reasoning
↓
Relevant Action
I kept customer memories isolated using customer-specific memory banks:
grocerai-customer-USER00001
grocerai-customer-USER00002
grocerai-customer-USER00003
This is important because personalization is only useful if the memory belongs to the correct customer.
A preference from USER00001 should never influence USER00002.
I explicitly tested this isolation rather than assuming it would work.
Memory Should Survive Language Changes
GrocerAI supports English, Telugu, Hindi, and mixed-language conversations.
That created an interesting test.
A customer can express a preference in Telugu:
"Naku brown bread ante chala istam,
white bread vaddu."
Later, they can ask in English:
"Which bread should I buy?"
The useful behavior is not literal sentence matching.
The agent needs to retrieve the underlying preference and connect it to the new request.
The same idea applies to other customer experiences, such as remembering lactose intolerance and later using that information when discussing alternatives such as oat milk.
That is where persistent semantic memory becomes more useful than keeping a giant conversation transcript.
The Customer Agent Also Understands Shopping Missions
Shopping is rarely just a sequence of independent questions.
A customer may have a mission:
Breakfast
├── Bread
├── Milk
├── Cereal
└── Fruit
The Customer Agent can work with that mission while the customer browses the store.
It can identify what has already been completed, what is still needed, and what actions can help the customer finish the mission.
This turns the interaction from:
“Ask a question → get an answer”
into:
“Start a shopping goal → receive assistance throughout the journey.”
Voice Uses the Same Agent
I did not want voice to become a separate intelligence layer.
The voice flow is:
Voice Input
↓
Speech-to-Text
↓
Customer Agent
↓
Backend Tools
↓
Reasoning
↓
Response
↓
Text-to-Speech
The same customer state, tools, mission, and memory can therefore be used whether the customer interacts through text or voice.
That keeps the architecture consistent.
From Big Store Display to Personal Companion
GrocerAI also separates the physical store experience from personal customer assistance.
The main store display handles shopping interactions such as browsing, searching, product information, cart actions, navigation, and checkout.
A customer can scan a QR code to activate a personal companion session.
That session gets a customer identifier and can provide personalized information such as:
- shopping mission progress
- remaining items
- recommendations
- notifications
- preferences
- availability
- personalized aisle guidance
- synchronized cart information
The important architectural rule is that the mobile experience is not a second independent cart.
It shares the customer's active shopping session with the store display.
The Complete Customer Learning Loop
The final Customer Agent loop became:
OBSERVE
↓
RECALL
↓
REASON
↓
ACT
↓
RE-OBSERVE
↓
REMEMBER
For example:
Customer says preference
↓
retain experience
↓
Later bread request
↓
recall preference
↓
Reason about suitable products
↓
Recommend relevant option
↓
Observe customer action
The important part is that memory is inside the decision cycle.
It is not simply a page showing stored memories.
What I Learned
1. An LLM should not be the database
Current inventory, cart state, mission state, and orders need deterministic application data.
The model should reason over that information rather than becoming the source of truth.
2. Memory should be selective
Not every conversation sentence deserves to become a long-term memory.
Useful memory is information that can improve a future decision.
3. Personalization requires isolation
A memory system that accidentally mixes customers destroys trust.
Customer-specific memory boundaries therefore became a first-class architectural requirement.
4. Multilingual support becomes more valuable with memory
Understanding Telugu or Hindi is useful.
Remembering a preference expressed in one language and applying it later in another makes the interaction much more continuous.
5. The real test is future behavior
I did not want to measure success only by whether the agent could store a memory.
The stronger test was:
Did recalling that experience change what the agent did next?
That became the core test for the Customer Agent.
Final Architecture
The Customer Agent ended up as a combination of:
Customer
↓
┌─────────────────┐
│ Customer Agent │
└────────┬────────┘
↓
┌─────────────────┐
│ Groq Reasoning │
└────────┬────────┘
↓
┌──────────┴──────────┐
↓ ↓
Backend Tools Hindsight Memory
↓ ↓
Current State Past Experiences
└──────────┬──────────┘
↓
Personalized Action
The result is not simply an AI that can talk about groceries.
It is an agent that can understand a shopper, use the actual store as its source of truth, remember useful experiences, and use those experiences when the customer returns.
That is the behavior I wanted from a personal grocery assistant:
not just answering the customer's next question, but becoming more useful because of what it learned from the previous one.
Top comments (0)