I’m building a small N-Back trainer — an exercise for working memory and attention.
During a training session, a square appears in different positions on the screen while letters are spoken aloud. The user needs to follow two sequences at once. If the square’s current position matches its previous position, press one button. If the letter repeats, press the other. Sometimes both signals match, and sometimes neither does.
When you spend a long time developing a product like this, its interface starts to seem obvious. You know what every button does, where to look, and when to press it. That’s why your first encounter with a real user quickly brings you back down to earth.
Version one: they’ll figure it out as they go
Initially, the tutorial was a short demonstration session. It showed the basic mechanics of N-Back but barely explained the controls.
I expected people to see the interface and figure out what to do next. In practice, some users couldn’t even work out how to start a training session. Others started their first session, pressed the wrong buttons a few times, and left.
Some people closed the page while still going through the tutorial. I decided the problem was unclear rules. That may have been the case, but I couldn’t prove it: my analytics showed users leaving but didn’t explain why.
Version two: now I’ll explain absolutely everything
My next attempt was the “Academy” — a separate section with theory, examples, and explanations of the controls.
The logic seemed reasonable: first, people take their time learning the rules, then move on to practice already prepared. But it ended up taking too long. Users skimmed through the theory and often never reached the point where they could finally try the exercise themselves.
I tried to solve the problem of an unclear tutorial with even more tutorial content. I think this was about when I started suspecting I was heading in the wrong direction.
Version three: action first, explanation second
After that, I decided to drop the long introduction and run the tutorial directly on the actual playing field.
First, the app shows only the square and teaches users to notice when its position repeats. Then the square is temporarily disabled, and the user works with sound alone. After that, both types of signals are combined.
A hint doesn’t just tell you which button to press. The app presents a predefined sequence of signals, knows the correct answer at each step, and waits for the user to take the required action. This means people aren’t reading instructions separately from the interface — they immediately try following them in practice.
The first steps use 1-Back mode (easy level): each new signal only needs to be compared with the previous one. This is the simplest version of the exercise. Once the hints end, an independent test consisting of 5 signals begins. At this point, the interface no longer highlights anything: users must determine for themselves whether the square’s position, the sound, both signals, or neither has repeated.
I like this version more than the previous ones. It gets people to their first meaningful action faster. But “I like it” still doesn’t mean “users understand it better.”
I opened Metrica to check my assumptions
To evaluate the new tutorial, I opened Yandex Metrica and looked at two goals: “Tutorial started” and “Tutorial completed.”
Over the week of September 8–14, 57 visitors started the tutorial, and completion was recorded for 26. But these numbers still don’t answer my main question: did it become clearer to people?
The goals themselves are configured correctly. The first triggers when the tutorial starts. The second triggers when someone has gone through the hints and finished the independent five-signal test.
But there’s a problem: finishing the test doesn’t mean completing it correctly. As the goals are currently configured, even if a user gets almost everything wrong, Metrica will still record the tutorial as completed. It turns out I measured tutorial completion, not an understanding of the rules.
My takeaways and what I’ll do next
I won’t delete the existing goals. They show how many people started the tutorial and how many reached the end.
I’ll also create a multi-step goal that tracks the user’s journey in order:
- Started the tutorial;
- Reached the end of the independent test;
- Started their first full training session;
- Finished it.
This goal will only trigger if someone completes all the stages in the correct order within a single visit.
I’ll add a separate event for those who actually pass the test without hints. Right now, Metrica doesn’t distinguish between completing it correctly and simply reaching the last screen.
My main takeaway is a little obvious: first, you need to define exactly what counts as success, and only then configure your analytics. Regular goals show individual actions; a multi-step goal shows the entire user journey. But even a multi-step goal will only tell you that someone went through all the stages. To understand whether they grasped the rules, you need to define and track a separate metric.
Here is the current version of the tutorial. If you give it a try, please let me know: was there a moment when you didn't understand what to do next? And what exactly was confusing at that point?
Top comments (0)