DEV Community

Raj Devnath
Raj Devnath

Posted on Originally published at Medium on

semlayer: build your semantic layer in minutes, so your AI can tap your warehouse’s hidden…

semlayer: build your semantic layer in minutes, so your AI can tap your warehouse’s hidden knowledge

Subtitle: Your company connected AI to its data — and nobody can tell if the numbers are right. The missing piece is not a better model. It is a thing most companies have heard of, almost none have, and nobody can afford to write by hand — which is why we built semlayer to work it out instead.

https://www.youtube.com/watch?v=FvQd4ZRzK14

Somebody in your company has pointed an AI at the data warehouse. Maybe it was sanctioned, maybe it arrived with a Copilot licence. Either way, people are asking it questions now.

This is a good thing. The queue of “can you pull me a number” requests that used to sit with three overworked analysts is suddenly answerable by the person who wants the answer. That is a real unlock, and it is why every company is trying it.

Then somebody asks for total revenue.

The AI writes a query. The query is valid. It runs in two seconds. It returns a number, confidently, with no error and no warning of any kind.

And it does not match what finance reports.

Nobody in the room can tell who is right

This is the moment that matters, and it is worth being precise about why it is so corrosive.

It is not that the AI made an obvious mistake. If it had crashed, or returned something absurd, you would laugh and move on. It returned a plausible number. Everything about the interaction said success.

So now you have two numbers and no way to adjudicate. The analyst who could have settled it is the person you were trying to stop interrupting. And the next time someone brings a number from the AI into a meeting, everyone discounts it a little — including when it is right.

That is how a tool dies inside a company. Not refused. Just quietly distrusted until nobody bothers.

It is not the model’s fault

The instinct is to blame the AI, or to wait for a smarter one.

We tested that. On a warehouse built to look like a real one, a small model got 58% of ordinary business questions wrong. A frontier model — a much more capable and more expensive one — got 45% wrong. Better, still unusable.

The reason is that the question was never really about intelligence.

Ask your best analyst why their revenue number is right and they will tell you something like: cancelled orders never count, and they are the ones with status code X. Ask where that is written down and they will say it isn’t. It is in their head, and in the heads of four other people, and in the fact that a summary table somebody built in 2019 only balances if you exclude those rows.

An AI reading your database can see the tables, the columns, and the data types. It cannot see the rule. There is nothing to see. The table definitions do not contain it.

So it does the reasonable thing: it adds up the column it was asked about, and it is confidently wrong by the exact amount of the rule it could not know.

The thing that would fix this has a name

What is missing is a layer between the raw tables and the person asking the question — one that records what the data means. Which rows never count. Which of the four order tables is the current one. What “a customer” is, given that the customer table keeps a new row every time somebody moves house.

This is not a new idea. It is called a semantic layer, and your data team has almost certainly brought it up.

Here is the part that does not get said out loud in the meeting where it gets brought up: every tool that offers one expects a human being to write it.

By hand. Table by table, rule by rule. On a warehouse of any real size that is a quarter of somebody’s life, and the somebody has to be one of the few people who actually knows the answers — the same people you cannot spare.

And then it has to be maintained forever, because the warehouse keeps moving. Someone renames a column on a Tuesday and a rule that was right is now silently wrong, which is worse than not having it.

So the project gets scoped, and costed, and deferred. Most companies never build one. This is not an indictment of anyone. It is a rational response to a job that costs a quarter up front and never finishes.

So build it the other way round

We built semlayer on a simple premise: most of that knowledge is recoverable from the data itself, without anyone writing it down.

Not all of it. But far more than people expect.

Point semlayer at a warehouse and it reads the data — not just the table definitions, the actual contents — and works out what the tables are, how they connect, and which rules the numbers obey. On the test warehouse in our short intro video that was 36 tables and 36 relationships, none of which had ever been declared anywhere.

And it found the cancelled-orders rule. Nobody told it. It noticed that a daily summary table only reconciles with the orders table when those rows are left out, then checked that this held in every store, every day, for three years before believing it.

That last part matters more than it sounds. The output is not a guess with a confident tone. Every rule it writes down comes with the evidence for it, so the person reviewing it can disagree.

Then the same revenue question returns the number finance reports.

Across those 38 business questions: 42% right from the raw warehouse, 87% with the layer. Same model. Same questions. The difference is entirely what the AI was allowed to know.

It takes minutes, not a quarter. And because it is inferred rather than hand-written, it can be re-inferred — so when somebody renames that column on a Tuesday, the layer moves with it instead of rotting behind it.

And then there is what the words mean

Suppose the AI now knows which rows never count. There is a second problem sitting underneath that one, and in most companies it causes more arguments.

Ask for “average order value”. Reasonable request. Your warehouse has a column of money, a table of orders, and a table of the individual lines inside those orders. Divide the money by the orders and you get one number. Divide it by the lines and you get another. On our test warehouse those two answers are $2,035 and $677 — three times apart, and both of them are defensible readings of what you asked for.

Neither is in your database. “Average order value” is not a column anywhere. It is a decision somebody at your company made, probably years ago, probably in a meeting, and almost certainly never wrote down.

This is why two people pull the same metric and arrive at the same meeting with different numbers. Nobody made a mistake. They used different definitions, and nothing in the system says which one is yours.

So semlayer records the definitions too — not just which rows count, but what the number is divided by and which filter belongs to it. Written down once, with the reasoning attached, so the next person to ask gets the same answer as the last one.

Which is the part worth taking away. Trust is not only about getting a correct number. It is about the same question returning the same number, to everyone, every time.

Where this does not help

One number we publish that most vendors would bury: on a clean, well-documented warehouse with sensible table names, our layer made things worse  — 67% from the raw tables against 58% with us.

If your warehouse is genuinely tidy, you do not need this, and we will tell you so. The value is concentrated exactly where the mess is, which is to say in almost every warehouse that has been running for more than a few years and has had more than one team touch it.

What to do with this

If your company is already trying AI on its data, the question is not whether the model is good enough. It is whether anything in your stack holds what your team knows.

If the answer is “it’s in people’s heads”, your AI is guessing, and the confidence of its answers is telling you nothing about whether they are right.

semlayer is free and open source. It runs inside your own environment, against your own warehouse, on your own keys — nothing leaves, and we never see your data. Your data team can have an answer about your actual warehouse this afternoon rather than after a procurement cycle.

The 90-second version of this argument, and the rest of the series: https://www.youtube.com/playlist?list=PLCrM1PMq6vVg

The code, the benchmark, and the numbers above including the one that looks bad: https://github.com/semlayer-dev/semlayer

Top comments (0)