Cosine similarity finds the topic, not the buyer

Saito — Sat, 11 Jul 2026 14:41:00 +0000

Cosine similarity finds the topic, not the buyer

I build a small tool called Leadgram that searches public Telegram groups for people who need what you sell. Up front, so nobody has to guess: it's my product, it's in beta, and this post is about the one problem that ate most of the build time.

It wasn't the search. The search was the easy half.

Two messages

Same Telegram group, same afternoon:

anyone got a CRM discount code?

we keep losing deals because our pipeline is a mess and nobody updates anything. there has to be a better way

If you sell a CRM, the second person is your buyer. They named a pain, they're frustrated, they're actively looking for a way out. The first one is hunting a coupon for a tool they already own and are perfectly happy with.

The task: build something that returns the second message and not the first.

Attempt 1: keywords

This is what every existing tool in this space does, and it's obvious why it fails. The buyer never types the word "CRM." A keyword alert for CRM returns the coupon hunter with 100% confidence and misses the person with the actual problem entirely.

Worse, keyword lists rot. You can sit there for an afternoon writing out pipeline, deal flow, spreadsheet hell, sales tracking — and the next buyer will phrase it in a way you didn't think of, because people don't consult your keyword list before typing.

The word isn't the signal. Move on.

Attempt 2: embeddings

The natural next move. Embed every message, embed the query, rank by cosine similarity. Maybe thirty lines of code. It feels like it should just work.

It doesn't, and the way it fails is the interesting part.

Query: "someone frustrated with their CRM and looking to switch"

Top results looked roughly like this:

0.89  great thread on CRM migration strategies, saving this
0.87  we moved off HubSpot last year, happy to answer questions
0.86  honestly CRMs are all bloated garbage
0.84  does anyone have a good CRM comparison spreadsheet?
...
0.71  we keep losing deals because our pipeline is a mess...   ← the buyer

Look at what happened. Every single one of those top hits is about CRM frustration. Cosine similarity did its job perfectly. The embedding model is not broken.

The problem is that the thing I actually want isn't semantic similarity.

What I want is a property of the speaker's situation: do they have this problem, right now, unsolved, and are they moving on it? Embedding space does not encode that. Consider four sentences:

"I had this problem."
"I have this problem."
"Someone should build a solution to this problem."
"I'm buying a solution to this problem on Monday."

They land in nearly the same neighbourhood. The vector captures the topic. It has no opinion about tense, stake, agency, or timeline — and those four things are the entire difference between a lead and a bystander.

There's a second problem stacked on top. A Telegram group is not a corpus of thoughtful statements. It's roughly this:

gm
gm ☕
+1
anyone catch the game
😂😂😂
[sticker]
is this group still active?

Ninety-something percent of it is social noise. Embed all of it and you don't get a search index, you get a very expensive way to retrieve the word "gm".

So there are really two jobs, and I'd been trying to do them with one tool:

Throw away the garbage. Most messages aren't signal of any kind.
Among what's left, separate the people who have the problem from the people who are merely talking about it.

The numbers, honestly

Twenty groups. Around 100,000 messages. That is the entire index.

That's small, and I'd rather say so than round it up to "millions." It also turned out to be the most useful property the thing has: at 100k I can actually read the failures. Every change to scoring gets checked against a set I hand-labelled myself, by eye, one message at a time. You lose that the moment you scale, and I don't think I'd have found the problem above without it.

Stack: Rust for ingest and API, Postgres for metadata, ChromaDB for the vector layer, TypeScript on the front end.

The part I don't get to skip

The messages get there because a Telegram account joins public groups and reads what's posted in them. That's scraping. I'm going to call it scraping, because half this category insists it's doing something else and it's tiresome.

What I think is worth defending isn't the collection — it's what happens after. A scraped list of everyone who typed "CRM" is worthless and a bit grim. A short list of people who genuinely asked for help, each with the sentence that made them match attached so you can check the machine's reasoning yourself, is a different object. The filter is the product. The scraper is plumbing, and it's the plumbing everyone else ships as the product.

And yes: this is a tool that can be used to cold-message strangers about things they said in public. I don't have a clean answer to that. The position I've settled on is that it reads public group messages only — never DMs, never anyone's account — and that if you're in an indexed group and want your messages gone, you email me and they're gone.  That's a person, not a policy page.

If you think that's insufficient, I'd rather hear it than not.

What I'd do differently

Start with the noise filter, not the search. I spent weeks tuning retrieval over a corpus that was 95% "gm", which is a bit like polishing a lens while the cap is on.

I write this stuff up as I go. The full workflow this turned into is here, and the thing itself is Leadgram — beta, free while it lasts, and small enough that I'll personally read your bug report.

DEV Community: Saito

Cosine similarity finds the topic, not the buyer

Cosine similarity finds the topic, not the buyer

Two messages

Attempt 1: keywords

Attempt 2: embeddings

The numbers, honestly

The part I don't get to skip

What I'd do differently