I started out playtesting world of goo as soon as we had something that was minimally playable, just to see if certain ways of displaying information made sense to people. It turned out that much of what World of Goo is now we owe to playtesting.
Just sitting quietly and watching people play the game was invaluable. People tell you more with body language than they ever would with words. I saw how people’s intuition around the game mechanic worked and adjusted the game to be more in line with what I saw people trying to do. As an engineer (and I know i’m not alone in this) I’m used to solving problems more by reasoning and my own intuition than by observation, so this was a new experience for me.
I tried to distill my approach down to a few guidelines that might help other developers improve their games:
1. Use only virgins – Two playtesting sessions with two different people (one session each) will give you a lot more information than two playtesting sessions with the same person.
2. Do it in person – The vast majority of useful stuff came by observing the players. Hardly anything came out of direct verbal feedback, so if you’re not there in person, the playtest is basically worthless. Sit where you can see the player, their hands, and the screen and just watch. There’s a lot to see.
3. Shut the hell up – I start a playtest session by saying “OK, this is the game. Play for as long or as little as you want. I’m not here.” Let’s be honest, you’re not going to be there when people play your game for the first time, so back off. Not saying anything is harder than it sounds. As the developer, you want people to enjoy the game, you want them to get it, you might feel frustrated when they don’t. You might feel the urge to say “no, just do this” or “try that” or “ignore this part”, but by suffering through these urges you get to see which obstacles are good challenges and which are products of bad design (you also reach a higher level of enlightenment, but that’s another subject entirely). The point is that you, the play tester, are a scientist and if you’re interacting with the player you risk contaminating your data.
4. Ask questions – The only time I regularly break rule #3 is when I see a player trying to do something I don’t understand. Without guiding them, find out what they’re trying to do, because at that moment they are following their intuition and an understanding of intuition is the game design gold you mine out of playtesting.
5. Take notes – What you don’t write down, you’ll forget. This should be a list of one liners. When you see something ugly in the game and think to yourself “Ick, I hope they didn’t notice that”, write it down. When the player seems confused, write it down (what might they be confused about?). If they’re trying to do something you hadn’t thought of, write it down. Any thought you have as you’re observing, write it down, nothing is too trivial, you’ll filter these notes later.
6. Follow through – For a single playtest session (for World of Goo it usually lasts between 30 and 90 minutes) I get up to two pages of notes. After a session kyle and I brainstorm possible changes and additions based on the notes we have. The end result is a todo list which I usually plow through pretty quickly (or at least file in our bug DB so that we don’t lose things).
Lather, rinse, repeat until your game is perfect.