Everyone's a BI Developer Now. Apparently.

AI can build your dashboard in minutes. Knowing whether the numbers are right is another matter.

I love AI.

I use it every day to write code, explore unfamiliar technologies, challenge my thinking and occasionally figure out why something I wrote three years ago makes absolutely no sense.
I'm also using AI to help write this article, which is rather ironic considering what I'm about to say.

But while I'm genuinely excited about what AI is making possible, I'm becoming increasingly uncomfortable with how quickly we're confusing the ability to do something with the ability to understand what we're doing. And nowhere is that becoming more apparent than in business intelligence.

Whatever happened to data literacy?

In my first article, I talked about Qlik's Data Literacy initiative and why I thought it was one of the better things the company had done.

I genuinely believed in it.

I was fortunate enough to have mentors who taught me about data, challenged how I approached problems and helped me build a career around something I initially knew very little about. Data literacy was about giving people the skills to question information, understand relationships and make better decisions.

Fast-forward to today, and we've got LLMs that can interrogate databases, generate SQL, create visualisations and explain the results without requiring someone to understand much about the underlying technology.

Recently, I watched a demonstration of Claude building dashboards and publishing them to Tableau. And I thought it was brilliant. Someone who's never touched Tableau can potentially create something useful in an afternoon. Someone learning BI can experiment without spending weeks figuring out which buttons to click.

I welcome that.

But the ease with which we can now create analytics has me wondering whether we're starting to undervalue the skills needed to understand them. Because learning how to build a dashboard was never supposed to be the point of data literacy.

AI is a skill multiplier

I've always maintained that AI is a skill multiplier. As a software engineer, I can use AI to build things faster, experiment with unfamiliar frameworks and learn concepts I might otherwise have spent days researching.

Sometimes it writes better code than I would have. Other times it produces absolute nonsense, and I spend longer explaining why it's wrong than I would have spent writing the thing myself. But I'm generally learning throughout that process. I can review the code, challenge decisions and recognise when something doesn't look right. And when I don't understand something, I have enough foundational knowledge to know what I need to investigate.

Now, give me the same AI tools and ask me to practise law. Would you trust me to represent you in court because I have a ChatGPT Pro subscription? I certainly wouldn't. So why should we approach business-critical analytics any differently?

Someone who doesn't understand data modelling, business definitions or even basic statistical concepts can now generate a dashboard that looks remarkably professional. That dashboard might even be correct. But would they know if it wasn't?

Using AI to accelerate learning is very different from using AI to avoid learning.

And I think that's a distinction worth keeping as these tools become more accessible.

Everyone's a BI developer now. Apparently.

With tools like Claude, Tableau Agent, Power BI Copilot and Qlik Answers, the technical barriers to analytics development are getting smaller. I've spent more hours than I care to admit formatting charts, dealing with obscure APIs and trying to explain why someone's preferred dashboard layout is a terrible idea.

If AI can take some of that work away, please do. But let's consider a fairly ordinary reporting scenario.

Imagine asking an AI assistant to build a sales dashboard.
It connects to your database, generates the queries and produces a rather impressive-looking dashboard showing that revenue has increased by 15%.

Excellent.

Except your business defines revenue differently from the way the model calculated it.

Perhaps cancelled orders weren't excluded. Perhaps customers were duplicated because of an incorrect join. Or perhaps the comparison includes a reporting period that shouldn't be there. The dashboard still looks fantastic. The charts work. The filters work. The numbers even reconcile with the SQL the AI generated. Unfortunately, the SQL is answering the wrong question.

Now, to be fair, some of the major BI platforms have already recognised this problem.

Qlik Answers can work with existing application logic and governed measures. Microsoft provides guidance on preparing semantic models for Copilot. These aren't necessarily systems handing an LLM unrestricted access to a production database.

Good.

That's exactly the direction these tools should be heading. But someone still has to define those measures, maintain the models and understand the business rules. And more importantly, someone needs to recognise when a technically correct answer is answering the wrong question.

We've spent years building semantic models, defining metrics, implementing access controls and establishing governance processes. I'd rather we didn't forget why we needed those things simply because someone demonstrated a particularly impressive dashboard in a five-minute video.

Trust, but verify

Anyone who's worked in software engineering should be familiar with this concept. We review code, write tests and validate assumptions before releasing applications into production.

At least, we're supposed to.

The fact that an LLM generated the code doesn't exempt it from review. So why would analytics be any different? If an AI-generated dashboard tells me revenue increased by 15%, I want to know;

• Where that number came from?
• What measure was used?
• Were the correct filters applied?
• Are the joins producing duplicate records?
• Are we comparing the same reporting periods?
• Who actually owns the definition of revenue?
• And can I inspect the calculation rather than simply trusting the explanation the LLM has given me?

This doesn't have to mean manually checking every number on every dashboard. But for business-critical reporting, somebody should be able to demonstrate that the calculations are correct and the underlying assumptions are appropriate.

Otherwise, all we've done is automate the creation of something we don't fully understand. Which brings me to another thing that's been bothering me.

Are we even asking the right questions?

Conversational analytics.

Qlik Answers, Tableau Agent, Power BI Copilot and the growing number of MCP integrations promising to let us talk to our data.

I like the concept.

Being able to ask questions in natural language instead of navigating through endless dashboards and filters is genuinely useful. And some products are already suggesting follow-up questions and helping users explore their data. But I've been thinking about the limits of that experience.

Let's return to our sales dashboard.
It tells me revenue increased by 15%.
I ask the AI assistant why, and it explains that sales increased in three regions, with one product category performing particularly well.

That sounds useful.
And it is.

• But what if the more important observation is that profitability has fallen?
• What if that revenue increase came almost entirely from one customer?
• What if the underlying data is incomplete?
• Or what if we're comparing two periods that shouldn't be compared?

A good analyst might recognise those possibilities and investigate further. Someone without that experience might accept the explanation and move on.

To be clear, modern AI systems can identify anomalies, suggest investigations and surface relationships that a human might miss. With the right context and analytical capabilities, they could do this exceptionally well. That's the part I'm excited about.

But there's a difference between an assistant that helps me investigate my data and one that simply provides convincing answers to whatever I happen to ask.

What happens when people don't know which questions they should be asking?

It's something I keep coming back to.

We've made it easier than ever to ask questions about data. I'm just not sure we've spent enough time considering whether that translates into better decisions. Sometimes we're simply giving people a more expensive way to confirm what they already believe.

Perhaps AI could actually make us more data literate

This is where I think there's a genuinely interesting opportunity. Imagine an AI assistant that doesn't simply tell me revenue increased by 15%, but challenges how I'm interpreting that result.

It might point out that one customer accounts for most of the growth. It might flag that profitability moved in the opposite direction. It might explain why the comparison is potentially misleading.

Better still, it could show me the underlying calculations and help me understand why those details matter. I'd be far more interested in an assistant that challenges my assumptions than one that produces another beautiful chart.

And perhaps that's how we should be thinking about AI and data literacy.

I don't want to discourage anyone from experimenting with these technologies.

Quite the opposite.

If someone can build their first dashboard in an afternoon rather than spending weeks learning a BI platform, I think that's progress. If an experienced analyst can spend less time formatting visualisations and more time investigating what the data means, even better. But I don't think we should confuse removing technical barriers with removing the need for expertise.

The tools are getting better. The opportunities are enormous.

I'd just like us to put the same enthusiasm into helping people understand the results as we do into helping them generate them.

Because ultimately, if AI gives everyone the ability to create analytics, who's making sure they understand what they've created?

And would they recognise when it's wrong?