DEV Community

Cover image for User-informed load tests
Floor Drees
Floor Drees

Posted on

User-informed load tests

June this year I joined k6. A week later I heard that we were being acquired by Grafana Labs. I was positively thrilled, I know Grafana as a great open source citizen. Anyway, k6 is in the performance space, and although I have some experience with testing (usability and accessibility testing), I've never really touched load testing until now.

There was (and is) certainly a lot to learn, and I can recommend checking out Nicole van der Hoeven's excellent videos on how to plan(1) realistic load tests(2).

One thing that struck me as odd though is that tools (like k6) that offer a browser recorder, promote it as a tool for testers, or other members of the QA team. As mentioned before I've done some usability testing and certainly so has my dad and he won't shut up about it. Now I don't often (or: never) listen to him, let alone quote him, but he says that once you've participated in a usability test, you're now a usability expert. What I think he means by that is that you know what's the expected outcome, and you can never look at the same product with fresh eyes again.

Having engineers potentially use the browser extension and have that be the foundation of your test script, for me, is like having the developer in the room while you're asking a potential user to perform an action. You can hear them think "how do they not understand how navigation works?!". As a side note: I sent engineers out of the room when they got fidgety or started to influence the user.

I guess what I'm trying to say is that I would expect to see teams invite users to perform a task and record their session(s). Better yet: different tasks (scenarios / user paths), combine those in your script as concurrent requests, and sprinkle some dynamic think time (rsleep();) on top.

And empty the cache after every session, to not muddy the results.

But of course that is usability testing, maybe perf or load testing really "just" needs to verify business requirements. While I don't think experienced testers would only test for the "happy path", I think others might. Especially as passing the test is a mandatory step in their build pipeline.

Please excuse the stream of consciousness, and you're hereby invited to yell at me in the comments when you think I'm totally off.

Top comments (0)