Introduction
As part of my day-to-day work as a data engineer, I've been building KPI dashboards more and more often.
While I have a fair amount of experience in data engineering—building data pipelines and setting up cloud infrastructure are my strong suits—KPI design was honestly never something I was good at.
In my past projects, there was always a BI engineer or a data analyst type of role on the team, so KPI design was handled by them through a division of labor. Being bad at it didn't really matter. Recently, however, I've increasingly been asked to handle everything end-to-end, from building the data platform to designing the KPIs, so that approach no longer works.
I was unfamiliar with it at first, but I've gradually figured out a way of doing things, so I'm writing it down in this article.
Case Study
My general approach is to start from a high level of abstraction and break things down top-down to decide on KPIs.
For example, in a past project, I worked on building a chat application. We ended up needing a KPI dashboard, and the first thing I did was to think about what the KPIs would be used for and who would be looking at them.
The company had to regularly request development budget from their executives, and they needed numbers to back up those requests—that was why they wanted the dashboard.
In that case, what you need is a dashboard that shows how much the system is actually being used. Typical examples would be the number of users, the number of chat messages, and the number of chat rooms. Then, what time axis should these numbers be plotted on? Executives want to invest in services that are growing, so showing month-over-month trends helps them understand how much the service is being used. Showing cumulative numbers is another option, but cumulative charts never go down, which makes it hard to tell whether the service is really growing compared to monthly trends. So cumulative charts are fine to have, but I believe they're insufficient on their own.
By showing trends in user counts and message counts like this, it becomes much easier to make a case such as: "This service is growing n% month over month, so we want to continue development. However, as the user base grows, cloud costs and personnel costs are also increasing, so we'd like additional budget to cover that." On the other hand, KPIs that merely look meaningful—like a per-user chat count ranking or user counts by role—may not be necessary in this case.
Thinking about KPIs this way also makes data modeling easier. For example, if the main view is monthly trends with cumulative numbers as an optional extra, you can store the data mart at a daily or event-level granularity, which lets you derive both cumulative and monthly views.
Let's consider another example: KPIs for product development. In that case, the main motivations would be things like: wanting to know whether marketing campaigns are working, how much impact a released feature had, or conversely, which features are not being used.
This company was running TV commercials, so we needed user trends before and after each TV commercial aired—in other words, the number of sign-ups per hour. For measuring the impact of a released feature, you need metrics tied to the expected effect of that feature, such as an increase in message counts or an increase in API requests for features available within messages. I think the same metrics can be used when deciding to remove unused features as well.
Conversely, the monthly user trends needed for executives may not be necessary in this case.
Alternative Approaches
That's how I've been designing KPIs. Since this is entirely self-taught, I took the opportunity to ask an AI for better approaches, and here's what it suggested:
1. Decision-driven design (recommended)
Work backwards from "Who looks at this dashboard, and what action does it change?"
For example, if "executives decide on budget allocation," go as far as asking:
what threshold would change that decision? Metrics that don't lead to action
can be cut as vanity metrics.
2. KPI tree (logic tree)
Pick one North Star Metric and decompose it into its components. For an AI
agent, for example: "weekly active conversations = number of users ×
utilization rate × conversations per user." With this decomposition, you can
pinpoint which factor is responsible when growth stalls, so it works for both
executive reporting and improvement analysis.
3. Persona × question list
For each type of user, first enumerate the questions they want answered.
"Executives: Is the investment paying off?" "PM: Did last week's release have
an impact?" "CS: Which customers are likely to churn?" Going in the order of
question → required chart makes the structure fall into place naturally.
4. Reverse validation: "delete it if nobody looks at it"
This is for after the dashboard is built, but instrument view logs and operate
under the assumption that any chart not viewed for 3 months gets deleted.
Dashboards only ever grow and rot, so it's good to build lifecycle management
into the design from the start.
One thing I'd add to your approach: tie an "owner" and a "target or threshold"
to each KPI. A dashboard that's just a wall of numbers ends up being something
people glance at and move on, but if it's decided that "when this number drops
below X, this person takes action," the KPIs actually start to function.
Working backwards from actions is something I often do when designing infrastructure monitoring. It goes like: "I want an alert when the system is down" → "How do we define 'the system is down'?" → "A spike in 500 errors, or RDB connections piling up, or etc..." It's the same idea.
Closing
That's all. This is just my personal approach, so there may be things I've gotten wrong, and there are surely other good ways to do this—but I hope it serves as a useful reference.
Top comments (0)