DEV Community: Christa

Hosting Successful Play Testing Sessions for Indie Games

Christa — Sat, 04 Apr 2020 01:55:46 +0000

I have been working as a team lead on an indie game for quite some time now, and have been working to implement some big quality of life changes and additional gameplay. The game in question, Kookie Kwest (trailer link: https://www.youtube.com/watch?v=gVYTaxtb02c) started off as a project in a course at the University of Toronto, Introduction to Video Game Design. It is a local competitive platformer game that supports from two to four players, where each player plays as a different colored knight and races to the top of a castle to collect a cookie (dubbed Kookie in the game).

The team consists of 21 members, all experienced in different fields of game development. We have sub teams focusing on development, game art, animation, and audio, all core aspects of the final game. Despite this large number of individuals contributing to our final product, this is still not enough! We still require play testers, typically unbiased individuals who will provide constructive criticism and critiques of your game. Play testing is extremely crucial to determine what areas your game performs well in, and where it falls short.

Throughout the development process, from the very first prototype of the game, to the version that we handed in as the final submission for the course, we have been working hard to gather as much feedback from as many people as we could. During these play testing sessions, we quickly learned what worked well during a play test, and what did not. In this post, I will be discussing a couple of dos and don’ts that I have collected over the course of the eight months developing this game.

Dos:

Create a survey.

Creating a quick survey for your play testers is one of the easiest ways to get feedback after a play testing session. Questions to include are scaled questions and multiple choice questions. An example of a scaled question is asking a user to rate the difficulty of the game on a scale of 1 to 10, with 1 being too easy and 10 being too hard. These types of questions limit the amount the play tester will need to write. Avoid bombarding the testers with too much text, as this can result in responses that are completed with little effort. Include a comments section at the end of the survey where the user can give any additional feedback that your survey did not address.

Create an in game tutorial.

Creating a tutorial at the beginning of the game can help the player understand what their objective is and learn the controls of your game. This is to test whether or not your tutorial on its own is sufficient without you having to explain to the users what to do. This is important as you will not be physically there to explain your game and the controls to every user upon the launch of your game!

Have regular play tests.

Even if your features are not all completed and perfect, it is important to host regular play testing sessions to identify big bugs and critical issues with your game early on that you may not have noticed. This will allow you to know where to allocate your time. For example, Kookie Kwest had a major platforming issue where the player would get stuck on surfaces. This was caught by many play testers early on. We dedicated two weeks to fixing this issue and pushed back our other tasks, as we saw this was more important. This will also help you reassess your goals and whether or not your game is meeting these goals.

Ask users to think aloud.

During the play testing session, ask your play testers to think aloud as they are playing. This will allow you to understand the thought process of the player as they are going through your game. This is important as users may not remember why they did something or made a certain decision in the game after the play test session is over.

Most importantly… Take notes during the play test session!

Take notes of what the players are confused about, what worked well, and what did not work. This will help later on when reviewing what changes should be prioritized (and also if you have poor memory).

Don’ts:

Do not point out bugs prior to the play test.

Even if you are aware a bug exists in your game, do not mention it to the play testers until after the play test session is complete. Telling the player about the bug before they begin to play could cause them to hyper fixate on the bug and fail to notice other things about your game. Instead, ask the user if they noticed the bug after the play testing session.

Do not pick biased play testers (or try not to).

Ideally, you should play test with people who you may be unfamiliar with, as this can reduce the amount of bias in their feedback. For example, your friends and family may feel inclined to give your game a positive review due to your relationship. Anonymous surveys may be able to mitigate this issue if you are hosting a play test session with enough testers. Ensure confidentiality within your play testers.

I hope this post was helpful in providing some insight into how to host a successful play testing session. I am open to answering any questions if you have any, and if you’d like to join as one of our play testers, please let me know!

Smart Watch Input Usability Test

Christa — Sat, 04 Apr 2020 00:38:25 +0000

For my last year of university, I chose to take a course on human and computer interaction (HCI) in order to learn more about usability and how to conduct proper experiments in HCI. In this course, I performed a couple of interesting experiments, one of which I will be summarizing in this post.

The experiment that I will be discussing is regarding text input on smartwatch devices. As smartwatches are becoming more prevalent, the need to come up with an input design that is able to perform efficiently and accurately becomes increasingly important. Smartwatch users want to be able to accurately input into their device without too much hassle. Some of the current input methods on the Apple Watch include a curated list of responses you can select from, or an area where you can draw characters onto the screen using your finger to send messages. These methods of input can be restrictive or even difficult as users are not given as much control as they have in a keyboard, and manually writing out each letter can be cumbersome.

The goal of my usability study was the test two potential methods of inputting text on the smartwatch. The first method consists of a QWERTY keyboard, the same as the keyboard found on the iPhone or your laptop, shrunken to the size of a smartwatch. I will be referring to this method as the “normal keyboard”. The second method was a zoom keyboard, where the user taps once to enlarge a certain area of the keyboard, and then again to select a character. My study focused on how accurate and fast the inputs were depending on what type of activity the user was doing (i.e. walking around or sitting down). Activity is an extremely important factor to consider, as many people use their smartwatches on the go.

Four participants were chosen to complete the experiment. The experiment consisted of 4 different trials – Using the normal keyboard while sitting down (sitting + normal), using the normal keyboard while walking (walking + normal), using the zoom keyboard while sitting down (sitting + zoom), and using the zoom keyboard while walking (walking + zoom). Each trial consisted of typing five phrases. The trials were put in different orders for each participant in order to assure that the results were not affected by the learning curve (i.e. they perform better on later experiments due to having been ‘trained’ by the previous experiments). The experiment was conducted on a mobile device that was strapped to the user’s wrist to simulate a smartwatch.

My predictions for the experiment were as follows, and they were all supported by the results of my experiment.

The zoom keyboard will have a higher average accuracy rate compared to the normal keyboard, as the zoom keyboard provides a bigger interface and more flexibility for errors.
The zoom keyboard will have a larger time per character compared to the normal keyboard on average, due to the fact that the user must click on the zoom keyboard twice for one input.
There will be greater change in accuracy between zoom and normal when walking than when sitting, due to the zoom keyboard providing more precision.

After performing the experiments, accuracy and speed (measured in seconds it took per character) were compared between each of the different trials. The zoom keyboard had an average accuracy of 77.5%, and normal keyboard had an average accuracy of 44.6%, indicating that the users were able to type more accurately using the zoom keyboard. Many participants expressed frustration with the normal keyboard setup, especially during the walking + normal keyboard trial due to the fact that it was extremely easy to make typos. This indicates that on average, the zoom keyboard is able to perform better.

The difference in accuracy between the trials is most noticeable between the walking trials, with the average walking + normal accuracy rate being 27.6%, and the average walking + zoom accuracy rate being 67.5%. There was also a greater accuracy rate in the sitting trials as well, although not as apparent, with sitting + normal having an accuracy rate of 67.9% and sitting + zoom with 87.5%. Both accuracies for either keyboard dropped when comparing external activity (sitting vs. walking), which highlights the importance of having an input method that is resilient to any external activity.

The change in accuracy between sitting and walking for the normal keyboard was 40.3%, whereas the change for the zoom keyboard was only 20%. This indicates that the external activity has an effect on how accurately the user is able to input the given phrases, with the normal keyboard being more affected than the zoom keyboard. This could be due to the shakiness when walking affecting the normal keyboard more, as the zoom keyboard provides more precision.

On average, the zoom keyboard (with an average of 1.06 seconds per character) took more than double the time of the normal keyboard (with an average of 0.486 seconds per character. This increase in time can be attributed to the fact that users must click twice on the zoom keyboard to input the desired character, compared to the normal keyboard, where they only need to click once.

Overall, the conclusions that can be drawn from this experiment is that keyboard type is significant in terms of activity, accuracy rate, and time it takes to input characters. The experiment also indicates that there may be a trade-off between time and accuracy, since the users were able to type faster on the normal keyboard, but more accurately on the zoom keyboard. Users were much more satisfied with their experience with the zoom keyboard due to the ability to accurately select their desired characters, regardless of what the activity was being done.

Personally, as the owner of an Apple Watch, I would really like to see some iteration of a zoom keyboard implemented, as the current methods of input are very limited. This was my first time conducting a formal usability test. I am happy to answer any questions if you have any!