The Playtesting Bible: A Guide for Game Developers

Playtesting is an incredibly common practice in the field of video game design. Without playtesters, the release version of a game would be nowhere near as high quality as the ones you see on shelves today. In this article I will look at why playtesters are important, how playtesting should be carried out and what makes a good playtester.

Why do we need playtesters?

Simply put: without playtesters, video games would be confusing, boring, and unbalanced. For instance, it might not have occurred to a group of designers that the instructions of their game are never explicitly written anywhere in the game’s first level, or perhaps certain elements of a puzzle aren’t as obvious as first anticipated, or even a particular enemy or item requires an explanation where you thought it didn’t. This all falls under the “confusing” section.

Playtesters sit through not only the disorientating parts of a game but also the boring parts. Sections of the game that are deemed uninteresting, overly punishing, or unrewarding are tweaked until a solution to these problems can be found. We’ve even seen players find completely unintentional alternate solutions to some of our puzzles! Finally, playtesters can also play with a game’s mechanics to ensure that it is balanced. By taking data from a large pool of playtests, a dominant strategy can be uncovered which can then be brought into line, allowing other strategies to emerge alongside it.

The champions with the highest win rate in League of Legends today.

It is worth mentioning that the most important part of playtesting is timeliness. Late playtesting data is as good as no playtesting data, because it means that the appropriate changes can’t be made to the game before release. The most common type of late playtesting comes in the form of reviews. Critics will say what is good and bad about your game, but by this point it is normally too late to act on the advice given. Organise your playtests early to ensure that your development team has enough time to analyse the data taken from tests and use it to improve gameplay.

On this note, I will stress that it is never too early to start playtesting. As soon as you have a functioning prototype, you can begin testing and seeing what works and what doesn’t. This can save a great deal of company time and money. As parts of your game are being introduced, playtest each one: each feature, each level, each character. It doesn’t matter if it’s the full playing experience just yet, as you’ll be testing that later as well.

Take, for example, The Sims. Will Wright originally envisaged The Sims as an architecture simulator, where the core gameplay revolved around building houses. The actual sims were simply there to assess your building when construction was complete. However, Maxis observed that players were really longing to see people living inside their newly-furnished homes, and thus The Sims became what it is today, the best-selling PC game in history.

The Sims (2000) changed from an architecture simulator to a life simulator during development.

It might frighten you as a developer to put your work out there, subject to scrutiny when in such a primitive state, but you needn’t be worried. It might not be finished, but this allows your team to see which areas of the game resonate best with the players. Your design team can then expand on these areas and develop them into core gameplay features. A playtest won’t give you all the answers, but it will certainly point you in the right direction, and it’s much easier to change direction after five paces than after five miles.

Playtests allow you to catch problems early, saving time and money in the long-run.

How should playtesting be carried out?

So now that we’ve established why we need playtesters, let’s move on to how to carry out playtesting sessions. As previously stated, playtesting won’t give you all the answers, but depending on how much data you can extract from a test, some sessions will be much more useful than others. Before you arrange any other aspect of playtesting, you should ask yourself what specific element of your game you want to test.

You might want to test the physics engine to see if your player can comfortably navigate difficult sections of parkour, or perhaps you need to check if a certain card feels overpowered in a trading card game, or you might see which character is the most appealing.

Hone in on the features you need to test: have your playtester try the level with the most jumping; limit their options to twenty cards and see which ten they pick; play short rounds instead of long ones to see how many times each character is selected. By knowing what you’re looking for, you can speed up the playtesting process and get more accurate and useful results. Bear in mind that sometimes you will just want your players to try the entire experience, but those sort of full-run playtests tend to come later in the development process.

If you are the one who is going to be coordinating and running playtests, make sure that you know your game intimately. You need to be on-board in case anything happens. If the player runs into a bug, how will they reset the game? Would they even be able to accurately write down the reproduction instructions? What if they get stuck on a certain section for the entire playtest? You need to have all the answers and all the solutions to these practical problems.

Bear in mind that this does not mean that you need to know how to fix every gameplay element immediately, but you need to make sure that the test can keep running in case something goes wrong. If you don’t know the answer to that puzzle on level one, any playtesters stuck on that section will not be able to progress past the beginning of the entire game. By the same token, you need to recognise if the game is working in ways other than intended that to a playtester would look completely normal and appropriate. If alien blood is meant to be green but is appearing red, or if no blood is showing at all, then you need to recognise that, as your playtesters won’t.

The very first level of Indiana Jones and the Infernal Machine, requiring you to pull this square out of the wall.

Although it sounds obvious, it is important to have reliable playtesters who actually turn up to playtests. If you have an hour slot booked for a player who doesn’t show up, waiting around wastes everybody’s time. It is just as important that your playtesters let you know if they can’t attend for some reason. Try to get the playtest to start as punctually as possible, because you may have other tests scheduled for the rest of the day and you don’t want them to run into each other.

With the preparation out of the way, we can now move on to the actual tests themselves. Try to conduct your playtests in as homely an environment as you can afford. Have a swivel chair for a PC game, or a sofa for a console game; allow them space to walk around in; provide them with refreshments. Noting when a player walks up to the fridge to pull out a drink can be important. It serves as a mini-break. Do they need a break on a level they are restarting a lot, or is it instead a reward for finishing a level? These breaks in gameplay are likely to be similar if the player were at home. Pause the game and go to the toilet, or wait until the stage is finished? This data could prove important to you.

One part of your player’s comfort that I haven’t yet mentioned is observation. When a player is observed, they will react differently to a game than if they had just installed it on their PC at home and fired it up. They know they are here to playtest, and that people will be watching them – not just any old people, but the designers of the game. This may result in a change of behaviour in the playtester.

They might be less inclined to react negatively to a part of the game that they didn’t enjoy for fear of hurting someone’s feelings. They might play with more focus and concentration than they would give a game usually. They might feel stress at the thought of having to do well and thus perform badly. This is why it’s important to minimise the effect of feeling observed as much as possible. Standing silently at the back of the room is a good start, but just being present might already be enough to distort your playtesting results. If possible, try leaving the room and allow your test subject to give themselves breaks, get themselves drinks and go to the toilet as and when they need to.

Players may react differently if they are constantly aware that they are being observed

Although you might want to explain your game in great detail before the playtester even gets the controller in his hands, resist the urge. See how well your game explains itself. Of course, if you are testing a very early prototype of your game some things will naturally have to be explained, but if they have any questions try to guide them instead of answering them out right.

A good response to “How do I turn the music off?” is “You can turn the music off – see if you can figure out how.” Observe which buttons they try to press, or where they look to do so. Is the M key the most obvious way to mute your game, or does your user dart straight to the options menu? This will reveal to you your playtester’s expectations. If they still don’t figure it out after about fifteen seconds, let them know so the playtest can proceed.

Be aware that sometimes the question will only come after having already looked for the option, so make sure to pay attention throughout the whole test.

This also brings us to our point about intervention. At what point should you intervene in a playtest? Whilst you don’t want to put the answers in the heads of your testers immediately, sometimes you will find it appropriate to give hints. If a player has been struggling on a certain battle for several attempts, or is hopelessly backtracking through an entire stage looking for an item, it might be an idea to put them back on the right track so as to save time.

Ask the player what they are trying to do so you can understand the problem at hand. You are not collecting data on how long it takes to backtrack the entire stage – the crucial piece of information here is that the player was unable to find what they needed and started to search elsewhere. Similarly, if an area of a game is broken, or if you know that the version you are using currently is outdated, there is nothing to be gained from testing that stage again. It is the new level that needs testing, not the older model.

Similarly, know when your playtesters are flagging. A particularly challenging, tough or draining level can severely impact the mood of your playtesters and make them less inclined to return. There is little fun in getting stuck in an area for a long time or repeating something several times over. If the player is so frustrated that they are considering rage quitting, it is best to help them through a section and deeply consider with the game development team why this section was so unappealing and possible solutions to, such as more forgiving checkpoints, weakening the bad guys, extending the time limit, etc.

The infamous barrel in Carnival Night Zone from Sonic the Hedgehog 3 (1994) was considered confusing by a huge number of players. Instead of bouncing up and down on the barrel, the player simply had to stand on it and press up and down. As such, this was removed from later versions of the game. Bear in mind that if a section is skipped and some tweaking is deemed necessary, this is a section you will have to test again in the future to see if the proposed changes have the intended effect.

It can be eye-opening to have several playtesters seated near one another during the tests. If a player has a question, they might find it more comfortable to simply ask the person right next to them. Just as important as the question is the answer given: has the other player successfully understood the game? “Which car is the best?” “The go-kart goes the fastest.” The go-kart may well have the highest acceleration, but that doesn’t necessarily mean it has the highest max speed. Despite this, it was perceived as the fastest car, which can influence your player’s choices. Pay close attention to players sat next to one another to see what aspects of your game they discuss.

Mental Block Gaming’s Templars of Hyrule (2014) is a three-player game. Listening to what the playtesters were saying to one another provided a lot of feedback

In addition to what they are saying, there are other ways to gauge a playtester’s thoughts. In an ideal world, you should set up a camera to record three things in a playtest: the tester’s face, their hands, and their monitor. The face can reveal a great deal of information about your game. It shows concentration, enjoyment, frustration, boredom, relief, and intensity. Laughing, smiling, frowning, sighing, gasping, even a little “Hmm…” can grant insight into what the player is thinking and feeling.

Whenever you write down notes about any part of the test, make sure to write in considerable detail. It’s easy to quickly jot down “Check wall” in the heat of a playtest, but two weeks down the line when collating data you might have no idea what this note is referring to. This is even worse than not noting the problem down, as you know there is a problem but you can’t identify it. Instead, “Check collision hitbox for buttons on wall on level 3” is a much better example of a useful note.

Once the test is over, you’ll want to discuss some aspects of the game to see what feedback your playtesters have. Take careful note of the order in which comments are delivered. The tester will prioritise what they feel was the most important aspect of their experience with the game by mentioning those points first. Also, bear in mind that there is often some disparity between what players say and what they feel. It is hard for anyone to accurately recount how they were feeling during each and every moment of an hour-long playtest, so make sure to use the data from the actual playtest along with the comments they make afterwards.

This is the part of the test where you have to put your pride to one side and remain calm and impartial. There will be criticism for your game. Players will bemoan parts of your game that you have spent months on. They will pull your baby to pieces, and you will instinctively want to jump in to defend it. You might have to get rid of features that you put a lot of work into, but remember that it’s all for the best. No single playtester’s experience is the be all and end all, but all of their feedback is valuable in some way.

On the other hand, no criticism at all is as unhelpful as not running the playtest in the first place. Let your players know that the whole reason they are here is to make the game a better experience for future players. By offering critique they are contributing to the final version of the game, so make sure that they don’t feel guilty for pointing out the bits they didn’t like. Remind them that they can be extremely honest with you, and that telling the truth is imperative to the job. If they don’t tell you about something that feels jarring, the reviews certainly will when the game is finally released.

It is not uncommon to receive several suggestions for improvement from your testers, but take these with a pinch of salt. Take note of the recommendations, but remember that the playtester is not part of the design team, and as such is unlikely to know what is best for the game. It is your job to sort through these suggestions and decide which ones hold any merit.

After players have given their general feedback on the positives and negatives along with their suggestions, it is good practice to get some discrete and empirical data on the playtest. Give your players a feedback form where they rate sections of a game on a scale of 1-5. Make sure to give each number a caption to help guide their usage, such as “Excellent” or “Poor”. An odd number of markers in your scale (1-5, 1-7, 0-10) is better than an even number (0-5, 1-10) because they give the user a definitive “middle point”. By collecting data in this way, you can then identify trends over the long-term to see if the changes being implemented are improving the gameplay experience or worsening it.

The Templars of Hyrule (2014) feedback form

Remember, you don’t need to explain yourself to your playtesters. You don’t need to talk about upcoming changes, things that you already recognise as problems, or problems that you found were only an issue for this one player. Simply acknowledge the points that they make and thank them for their time.

Who should your playtesters be?

Let it be said that some people will make better playtesters for your game than others. Firstly, your group of playtesters should include a majority of members from your target audience. If your game is designed for boys, then girls are less likely to provide useful feedback, and vice versa.

That said, having a variety of playtesters for your game is certainly not a drawback, and can often provide useful insights into how you should market your game. Sometimes players that you didn’t expect to enjoy the game are the ones who enjoy it the most. If we look at My Little Pony, for example, the franchise’s target market is girls; however, an unanticipated Brony culture developed from the animated television series since the content unexpectedly appealed to the demographic of adult males.

The animated television series My Little Pony: Friendship is Magic (2010)

Remember that a good playtester isn’t necessarily somebody who can complete your game easily. In fact, it is the ones who struggle that will expose your game’s weaknesses. Arguably, the worst person to playtest your game is you, since you know exactly how your game works: you know every enemy, every weapon, every level, and every puzzle inside out. (That said, if you don’t, you should probably be better informed before leading any playtests.) You are biased towards your own game, and even if you claim not to be, the experience of one person alone does not stand for the majority. This is why it’s important to have other people try out your game.

A playtester doesn’t necessarily have to be someone who is good at games. There is much more to be learned from a gaming novice than a gaming veteran who has a great knowledge of general video game conventions. Avid gamers are likely to be able to quickly pick up a game without any instructions through the process of trial and error by making educated guesses based on other games that they have played. For example, character movement in many PC games is controlled by right-clicking the ground, such as in DOTA or Age of Empires. We, as gamers, take this for granted. Left-clicking would perhaps seem more natural to some.

Playtesters who are less experienced will have to be able to learn the game without too much frustration in order to enjoy the experience, so if any instructions are not clearly stated they will likely falter when attempting to perform those actions. By carefully observing players who are unfamiliar in a gaming environment, you can ensure that casual players and children will be able to understand and access your game.

The WASD keys are often used to move, although the arrow keys might seem more obvious

That said, make sure to include players that are in your target audience in your playtests. If you are being told by the younger audience that cut-scenes don’t make sense or that the game is too hard, take this with a pinch of salt. An older player might fully understand what’s going on, and might think the difficulty is in line.

On the other hand, if older players find a children’s game silly or easy, that might mean you’re on the right track. Using the wrong playtesters for your game can be counterproductive. Having the elderly test the latest shoot-em-up and taking their suggestions might lead to an incredibly diluted game; however, there are always exceptions to the rule, as demonstrated by GAMERGRAN87’s killing spree in Grand Theft Auto V (2013).

Warning: This video contains offensive language

It is paramount to the health and safety of your playtesters that you check if they have any conditions that would affect their ability to play your game. Ensure that your testers are well aware of all dangers involved in your game, and alert them of any sensitive material they may encounter whilst playing. You should let your playtesters know if your game contains any of the following:

• Flashing images
• Jump scares
• Blood and gore
• Profanity
• Nudity
• References to drugs, sex or gambling

It only takes a few seconds to check, and it’s much better to know than to risk upsetting or triggering your playtester.

Over the course of your playtests, you will have loyal testers that become ‘veterans’ of the game. Remember that these players are much more likely to understand even previously untested areas of a game because they know the game so intimately. For example, in Mental Block Gaming’s Templars of Hyrule (2014), our rendition of the Bottom of the Well plunges the player into pitch-black conditions.

We tested the dungeon before this feature had been added, so when a returning group of playtesters tried the map for a second time they already knew the layout of the dungeon despite having reduced vision in the second playtest. This is true for even the most basic of things: pressing three buttons at once is a very common feature in our three-player maps, but it sometimes takes players a few minutes to realise that they need to press these buttons simultaneously. After a few dungeons, players become accustomed to the playstyle of your game, which means that they will henceforth be unable to test the effectiveness of the on-boarding process for your game.

The lit version of the Bottom of the Well from Templars of Hyrule (2014)

To conclude, playtesting is important because it shows whether or not your game is accessible and fun. Letting your testers play in a comfortable environment where they don’t feel observed will give more accurate results. You should intervene as little as possible, only guiding the player when they are struggling a lot or when they are on the wrong track. Carefully recording the comments and actions of your players allows you to accurately interpret their responses. Finally, understanding how different types of playtesters affect the feedback that you receive enables you to decide which changes are necessary.

[googlead]