This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
Picture a small thrift page on Instagram. The seller posts a kurti at night, and by the time she's finished dinner there's a pile of DMs waiting. "Size kya hai?" "Chest kitna hai?" "Price final?" "Pune tak kitne din lagenge?" Same four questions, from different people, and each one needs an answer or the buyer moves on to the next page.
I haven't sat next to a seller while this happens, so I'm not going to call it a case study. It's the scene that started this project, and it's a scenario, not a story from a real shop.
Jawab Desk ("jawab" means reply in Hindi and Marathi) is my attempt at the boring middle part. The seller fills in an item sheet once: price, size, flaws, delivery. When a buyer messages, a small open model drafts a reply in the buyer's own language, whether that's English, Hinglish in Roman letters, or Marathi in Devanagari. It's only allowed to use what's on the sheet. If the answer isn't there, it should say the seller will confirm. After that, plain code checks the reply and blocks it if it contains a number or a word like refund, warranty or discount that the sheet never mentioned.
Would a seller pay for this? I think so, but that's a guess. I haven't tried it with a real seller yet, and my test messages are synthetic. Keep that in mind for everything below.
Demo
There's no hosted demo. I built it to run on the seller's own laptop. python app.py opens a small Gradio page, and JAWAB_MOCK=1 python app.py shows the interface without downloading a model.
Code
How I Built It
The models are Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct, run through Hugging Face transformers on a laptop CPU. No GPU, no hosted API. I wanted to test the 3B model as well, but the 1.5B one already took about 80 seconds per reply, so I dropped it.
The model gets a simple rule: answer only from the item sheet, and if the answer isn't there, output exactly [CONFIRM]. I don't trust the model to follow that every time, which is why the second layer exists. After the model replies, code pulls out every number and a short list of claim words and checks them against the sheet. If something doesn't match, the reply gets swapped for a fixed "seller will confirm" message in the buyer's language.
There are 37 tests for the guardrail, language detection and the scoring code. I built this with an AI coding agent and checked the results myself. Its first full run crashed on a bug when unloading a model, so it fixed that and ran everything again from the start.
How It Did
I wrote 24 test messages across three made-up item sheets: a thrifted kurti, a homemade cake order, and a handmade earring set. Twelve could be answered from the sheet and twelve couldn't. They're in English, Hinglish and a few in Marathi, and they were written by the coding agent, not taken from real chats.
| Model | Answerable questions answered correctly | Unanswerable questions where the model itself said "seller will confirm" |
|---|---|---|
| Qwen2.5-0.5B-Instruct | 67% | 50% |
| Qwen2.5-1.5B-Instruct | 75% | 8% |
The bigger model was better at using the sheet, but it almost never admitted when it didn't know something. That's the worst kind of mistake for a shop, because a confident wrong answer is worse than a slow right one.
The three worst replies all came from the 1.5B model, and all three got past my guardrail:
- Asked for the seller's other listings, it made up an Instagram handle.
- Asked "Do you accept credit cards?", it said yes. The sheet never mentions cards.
- Asked about international shipping, it invented a policy and extra shipping costs.
The guardrail only catches numbers and a fixed word list, so it misses invented facts that contain neither. I wouldn't call this system safe. And 80 seconds a reply on my CPU is far too slow for real DMs. The 0.5B model also failed the international shipping question.
Why Does Open Innovation Matter?
For a seller, the appeal is practical. After the one-time download it runs on a normal laptop with no internet and no account. There's no per-reply fee, which matters when you're answering hundreds of messages for small margins. Buyer messages, addresses and prices stay on the seller's own machine.
It also let me look inside the problem. I could pick the model size, compare the two, and put my own check around the model. That's how I found out the bigger model was more confident, not more careful.
It's slow, and it still makes things up now and then. But it runs on a laptop, it costs nothing per reply, and I can now see exactly where it fails. The next step is a real seller.
Top comments (0)