DEV Community: Ilia Ivanov

I built a site chatbot that isn't allowed to make things up

Ilia Ivanov — Fri, 16 Oct 2026 07:07:45 +0000

In 2024 a tribunal in Canada made Air Canada honor what its website chatbot had told a customer about refunds. The airline argued the bot was responsible for its own words. The tribunal said the company is responsible for everything on its site, chatbot included.

For a small business that is the real problem with site chatbots. A wrong opening time or an invented price costs more than the bot saves. So I built a demo where the model is only allowed to draft, and plain code decides what the customer sees: Front Desk.

The setup

The demo runs on Juniper Street Bakery, a bakery I made up. Its whole "site" is 13 short entries: hours, prices, cakes, allergens, delivery, payment. The assistant knows nothing else.

The model is Gemma 4 E2B, running in the browser through WebLLM at temperature 0. Every question goes through four steps:

Find. Code picks up to three entries that match the question. If nothing matches, the model is not asked at all and the customer gets the phone number.
Draft. The model reads only those entries and returns JSON: the answer, the id of the entry it used, and the sentence it took the answer from.
Check. The quoted sentence has to be in that entry. Every number in the answer has to be in it too ("4 pm" counts as 16:00, "$4" does not count as $4.50). And every word that carries meaning has to appear somewhere in what the model was given.
Show or step back. If the first sentence fails a check, the draft is thrown away and the customer sees the entry's exact text or the contact details. Later sentences that fail are cut.

What the first runs showed

I wrote 24 test questions with known answers, 6 of them about things the site never mentions (pizza, a loyalty card, Christmas hours).

Run 1: 17 of 24. No invented fact reached the customer, but most misses had the same cause. Asked "Do you deliver bread?", the model set answerable: false, because to it "no" felt like "not answerable". Rewording that rule fixed it.

Run 2: 21 of 24, and a real hole. Asked "Do you have a loyalty card?", it answered "No, we do not have a loyalty card." The site says nothing about loyalty cards. There were no numbers in that sentence, so my number check let it through.

That is where the word check came from: "loyalty" is not in anything the model was given, so the sentence is not backed by the site. The same check later cut "That is our capacity." from an otherwise correct answer about seats.

On questions I hadn't tuned on

After writing the checks I added 7 new questions and ran all 31 live. It got 28 right, 5 of the 7 new ones. One miss was my own bug: the model wrote "$8.00" and the site says "$8", and the number check treated them as different. With that fixed, the same recorded drafts pass 29 of 31.

The two misses left are worth showing, because the checks can't catch them:

"Is the bakery open on Christmas Day?" got the regular opening hours. True, every word on the site, but not an answer.
"Is the cinnamon roll vegan?" got a correct "no", but pointed at the pastry prices instead of the vegan entry.

A wrong yes or no where every word is on the site would pass the same way. That is why I think a bot like this needs a test set of the business's real questions before it goes live, not only a good prompt.

Do small businesses even have chatbots?

I took 1,020 business websites in Austin from OpenStreetMap (cafes, restaurants, dentists, clinics, salons, bakeries), dropped chains that share one domain, and fetched each homepage once. 592 loaded. 27 had any chat widget at all. 15 of those were Wix's built-in chat with the owner. The only ones I could identify as AI assistants were on chain sites.

So the chatbot market is crowded, but the local cafe mostly has no assistant at all.

A widget without a model

That led to a second mode. The widget is one script tag. Without a model endpoint it runs in quote mode: it finds the entry that matches and shows it word for word, or gives the phone number. It cannot make anything up and costs nothing per question. On the 31 test questions it shows the right entry or declines 29 times, though I also used those questions while choosing search words, so expect less on a new site.

With an endpoint added, it drafts answers and runs the same checks, falling back to quote mode when one fails.

Try it

Demo with recorded answers, no download: https://phazww.github.io/front-desk/#recorded
Live model and the test set: https://phazww.github.io/front-desk/#tests (needs WebGPU, downloads the model once)
Code: https://github.com/phazww/front-desk

I set this up for small businesses on their own content, with a test set of their customers' questions. Cafes and bakeries first: https://phazww.github.io/front-desk/cafes/

What would you check that I'm not checking?

Email verifiers tell you an address exists. I built one that tells you if you may write to it

Ilia Ivanov — Thu, 15 Oct 2026 11:24:10 +0000

Apify Store has a lot of email verifiers. The biggest ones I found had between 400 and 1,400 users in the last 30 days, at about $0.90 per thousand addresses. They answer one question: does this mailbox exist?

None of them answered the question I kept running into: am I allowed to send a first marketing email to it?

That answer depends on the country of the person you are writing to. Germany, Spain and Italy want consent before the first email, even to a company address. The US allows it, but wants a postal address and an opt-out in every message. Sweden protects people and not companies. Slovakia and Slovenia care whether the company published the address itself.

I had already read those laws for 31 countries for a free web checker. This week I turned the same rules into an Apify Actor: Cold Email Law Checker.

What it does

You give it a lead list. It adds a verdict to every row:

law_verdict: ok, cond (only in specific cases), consent or unknown
law_reason: one sentence on why
law_mustInclude: what the email itself has to contain, such as an opt-out or a postal address
law_lawSource: a link to the law or the regulator
law_checked: the date I checked that country

Your own columns stay, so you can filter the dataset and keep only the rows you may email.

Three ways to feed it

The output of another Actor run. Pick the dataset of your lead scraper. Nested fields like location.country are found automatically.
A CSV link.
A JSON list of addresses or objects.

It works out which field is the email, which is the country and which says where the address came from. You can name them yourself if it guesses wrong.

Two details that change the verdict

Use the person's country, not the company's. The rules follow the recipient. A German employee of a US company is a German recipient. When a row has both, the checker prefers the person's location.

Say where the address came from. In Canada, Australia, the Netherlands, Slovakia and Slovenia the answer depends on whether the company published the address itself. Add a source column: published, or vendor for anything a tool found or guessed. Values like apollo or hunter count as vendor.

How I checked it

The Actor and the web checker share one rules file, generated from the same data, so they cannot drift apart. The tests run an example export through the Actor and compare every verdict with what the website says for the same rows. Running the example input before launch caught a bug, and now there is a test for it: the Actor looked for the source column only in the first row, so a Slovak address published on the company's own site got the wrong verdict whenever the first lead had no source.

On my laptop it checks 20,000 leads in about four and a half seconds. On Apify the first runs took one to two seconds.

Limits

This is a research summary, not legal advice. It covers the first unsolicited marketing email to a business recipient. GDPR applies on top whenever an address identifies a person. Every verdict links to its source so you can check it, and if you know one of these laws better than I do, open an issue on GitHub and I will fix it.

It is free while I find out whether people use it.

I read the cold email law of 30 countries. Writing to info@ needs consent in 10 of them

Ilia Ivanov — Wed, 14 Oct 2026 11:00:11 +0000

Last week I almost sent cold emails to small shops in Spain. Then I read the Spanish law and found out a first marketing email there needs consent, even when you write to a company. Vendor blogs said different things, so I went to the source: the law text or the regulator's own guidance, one country at a time.

I am at 30 countries now. Here is what they say about the first unsolicited marketing email to a business. This is a research summary, not legal advice.

Writing to a generic company address (info@company)

No consent needed (14): United States, United Kingdom, France, Belgium, Ireland, Sweden, Norway, Finland, Portugal, Estonia, Latvia, Luxembourg, Slovenia, Croatia.

Allowed with conditions (5): Canada, Australia, Netherlands, Slovakia, Bulgaria.

Consent first (10): Germany, Spain, Italy, Denmark, Austria, Poland, Switzerland, Czech Republic, Malta, Greece.

Not verified yet (1): Cyprus. The consent rule there covers only natural persons, and companies fall under a separate 2005 order I have not been able to read.

Writing to a named person (jane.doe@company)

This is where it gets strict. Plainly allowed without consent in only 4: the US, the UK, France (if the offer relates to the person's job) and Sweden. In several countries the law protects "natural persons" and says nothing clear about an employee's work address, so for 7 countries I marked it as not verified instead of guessing.

Things that surprised me

Belgium treats info@ and jane@ differently. A royal decree lifts the consent rule for legal persons only when the address is impersonal.
Slovakia has an unusual exception: no consent is needed when you write to contact details that a company or sole trader has published itself (section 116(15) of the Electronic Communications Act).
Slovenia says almost the same thing about named people: you can use a person's address if the company published it as its own contact address (article 226(6) of ZEKom-2).
Bulgaria keeps an official opt-out register for companies. You can email a company unless its address is in the register, and the email has to be recognisable as unsolicited as soon as it arrives.
Greece and Malta protect companies exactly like individuals. Consent first, for everyone.
The US is the opposite: CAN-SPAM is an opt-out law and covers B2B email, but every email needs a postal address and a working way to opt out.

What every country asks for, even where consent is not needed

Say who you are, and give a working way to stop. Most of the countries above put that in the law text itself.

The data

Every country has its own page with the verdict for each kind of address, what the email must contain, and links to the law or the regulator. Each page shows the date I checked it.

https://phazww.github.io/can-i-cold-email/

If you know one of these laws better than I do and see a mistake, tell me in the comments and I will fix it and credit you. The next countries I want to add are Lithuania, Hungary and Romania. Their official law portals were not reachable from where I am, so if you have a link to the current text, that helps too.

Cold email law changes at every border, so I put an open model in the browser to check my drafts

Ilia Ivanov — Tue, 13 Oct 2026 11:40:01 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I'm 18, I live in Shenzhen and I do freelance web work. Last week I started sending cold emails to small online shops in Europe. Then I read the Spanish law and found out that a first marketing email there needs consent, even when you write to a company. I stopped and went to read the rest.

I read the rules for 20 countries, one law at a time, and I am up to 28 now. They do not agree with each other. Germany, Spain and Italy want consent first. France lets you write to a professional if the offer relates to their job. The US wants a postal address and a way to opt out in every email. Sweden protects people but not companies.

Coldproof checks a cold email draft against the law of the country it is going to.

You pick the recipient's country and the kind of address, and paste your draft. You get three things back:

The legal verdict for that country, with a link to the law or the regulator.
A checklist of what that country requires inside the email, with the exact words the model found in your draft for each item.
A fixed version with the missing lines added, in the language of your email.

Who would pay for it: sales teams and outbound agencies that send to several countries at once. One wrong campaign can mean a fine or a burned domain, and nobody on the team has time to read 28 laws. A single draft is free. The page has a "for teams" offer at $19 per seat with a button that only counts interest, because I wanted to know if anyone cares before I build billing.

Demo

https://phazww.github.io/coldproof/

On a phone, or if you do not want to download a model, press "See a recorded example". It replays a saved answer from Gemma on the sample draft through the same checks, and it says so on the page.

On a computer, click "Try a sample" and then "Check my draft". The first run downloads the model once. After that a check takes about 10 seconds on my Mac with Gemma, and about 6 with the smaller Qwen model.

It needs WebGPU, so use a recent Chrome, Edge or Safari on a computer.

Code

phazww / coldproof

Check a cold email against the law of the recipient's country. An open-weight model runs in your browser, the draft never leaves your machine.

Coldproof

Check a cold email draft against the law of the recipient's country. An open-weight model runs in your browser through WebGPU, reads the draft and reports what it contains, quoting the exact words. The page compares that with checked rules for 28 countries and returns the missing pieces and a fixed version. The draft never leaves your machine.

Live: https://phazww.github.io/coldproof/

How it is built

rules.js: per-country rules for the first unsolicited marketing email to a business recipient, each with its source. The research comes from my earlier project can-i-cold-email, checked against the law or the regulator on 10 and 11 October 2026.
app.js: loads the model with WebLLM, asks it for a JSON report with quotes, verifies that every quote is really in the draft, and assembles the fixed version from hand-written wording in 13 languages.
Two open-weight models to choose from: Gemma 4 E2B…

View on GitHub

No server, no API key, no build step. Three files: the rules, the page, and the script.

How I Built It

The default model is Gemma 4 E2B, running locally in the browser through WebLLM on WebGPU. There is a second option, Qwen 3.5 2B, which is about 1 GB and quicker to get. Both are open-weight.

The first version let the model do everything: read the draft, decide what was wrong, rewrite it. It looked great on the English sample. Then I tested a French email. The model was asked to add an opt-out line and wrote, in polite French, "if you would like to receive information about our services, please reply to this message". That is the opposite of an opt-out. For a tool about the law, that is the worst possible bug, because it looks correct.

So I took jobs away from the model until it had only the one it is good at:

The model reads. It gets the draft and returns a JSON report: the sender's name, the sender's company, whether there is a way to say stop, a postal address, a plain commercial offer, and what language the email is in. For every "yes" it must quote the exact words. WebLLM's JSON schema mode keeps the output in shape.
The page checks the quotes. If the model says "yes, there is an opt-out" but the quoted words are not in the draft, the page counts it as missing and says so. The quoted sentence also has to look like an opt-out. A small model cannot talk its way past this.
The law is not the model's job. Which elements each country requires comes from a rules file. I built that research last weekend for another project of mine, can-i-cold-email, and checked each country against the law text or the regulator. The app in this post was built this week on top of that data.
The legal lines are fixed wording. The opt-out sentence and the other added lines exist in 13 languages and are fixed in advance, not generated when you run a check. The model only tells the page which language the email is in. I do not speak all 13, so corrections are welcome.
The subject check is plain code. A first email that starts with "Re:" is pretending to be a reply. That does not need a model.
Consent countries get a stop sign. If the country requires consent before the first email, the page says so and does not offer a rewrite. No wording fixes that.

Then I wrote a proper test set: 30 drafts in 13 languages, each with the answer a careful human would give. Some are traps: "unsubscribe" used in another sense, a signature with only a first name, "The Invoicely team" with no person, a city with no street address, a vague question with no offer, an opt-out hidden in a P.S.

Early on, the smaller Qwen model accepted the signature "Sam" as identifying the sender, with no company anywhere, and once pointed to "We make switching free." as the opt-out. So the sender check now asks for the name and the company separately, and the opt-out quote has to contain opt-out wording.

The first full run gave Gemma 28 of 30. Both misses were mine. Gemma found the opt-out in "just reply 'no thanks'" and "let me know and I won't follow up", quoted the right words, and my own second check threw the answer away because it did not know those phrasings. I widened the wording list, checked it still rejects sentences like "We make switching free.", and ran everything again.

Model	Drafts fully right	Seconds per draft
Gemma 4 E2B (default)	30 / 30	about 10.5
Qwen 3.5 2B	26 / 30	about 7.4

Qwen still misses two opt-outs phrased as "let me know", calls a vague question a sales pitch, and calls one French pitch not commercial. That is why Gemma is the default. Anyone can rerun the set in their own browser at phazww.github.io/coldproof/#tests.

I worked with an AI coding agent the whole way. The bug with the French sentence was found by testing real drafts in the browser, not by reading the code.

Why Does Open Innovation Matter?

A cold email draft is a list of things a company does not want to leak: who they are targeting, what they are offering, sometimes a customer's name. A lead list is personal data. The whole point of this tool is to help people respect privacy law. Sending that data to someone else's API to check it would be a strange way to do that.

An open-weight model lets the check run on the user's own graphics card. Nothing is uploaded. There is no account, no key and no server for me to pay for, which matters when you are 18 and your hosting budget is zero. It also means the tool keeps working if I disappear.

And open weights let anyone see exactly which model made the call, swap it for a better one, and test it on their own language. A closed API would have hidden the French bug behind a nicer average.

Prize Categories

Overall
Best Use of Gemma: Gemma 4 E2B is the default model and runs locally in the browser. It got all 30 test drafts in 13 languages right.