Ron’s Rules for Playtesting

I started out playtesting world of goo as soon as we had something that was minimally playable, just to see if certain ways of displaying information made sense to people. It turned out that much of what World of Goo is now we owe to playtesting.

Just sitting quietly and watching people play the game was invaluable. People tell you more with body language than they ever would with words. I saw how people’s intuition around the game mechanic worked and adjusted the game to be more in line with what I saw people trying to do. As an engineer (and I know i’m not alone in this) I’m used to solving problems more by reasoning and my own intuition than by observation, so this was a new experience for me.

I tried to distill my approach down to a few guidelines that might help other developers improve their games:

virgin.jpg1. Use only virgins – Two playtesting sessions with two different people (one session each) will give you a lot more information than two playtesting sessions with the same person.

2. Do it in person – The vast majority of useful stuff came by observing the players. Hardly anything came out of direct verbal feedback, so if you’re not there in person, the playtest is basically worthless. Sit where you can see the player, their hands, and the screen and just watch. There’s a lot to see.

shutupfool.jpg3. Shut the hell up – I start a playtest session by saying “OK, this is the game. Play for as long or as little as you want. I’m not here.” Let’s be honest, you’re not going to be there when people play your game for the first time, so back off. Not saying anything is harder than it sounds. As the developer, you want people to enjoy the game, you want them to get it, you might feel frustrated when they don’t. You might feel the urge to say “no, just do this” or “try that” or “ignore this part”, but by suffering through these urges you get to see which obstacles are good challenges and which are products of bad design (you also reach a higher level of enlightenment, but that’s another subject entirely). The point is that you, the play tester, are a scientist and if you’re interacting with the player you risk contaminating your data.

4. Ask questions – The only time I regularly break rule #3 is when I see a player trying to do something I don’t understand. Without guiding them, find out what they’re trying to do, because at that moment they are following their intuition and an understanding of intuition is the game design gold you mine out of playtesting.

notes.png5. Take notes – What you don’t write down, you’ll forget. This should be a list of one liners. When you see something ugly in the game and think to yourself “Ick, I hope they didn’t notice that”, write it down. When the player seems confused, write it down (what might they be confused about?). If they’re trying to do something you hadn’t thought of, write it down. Any thought you have as you’re observing, write it down, nothing is too trivial, you’ll filter these notes later.

6. Follow through – For a single playtest session (for World of Goo it usually lasts between 30 and 90 minutes) I get up to two pages of notes. After a session kyle and I brainstorm possible changes and additions based on the notes we have. The end result is a todo list which I usually plow through pretty quickly (or at least file in our bug DB so that we don’t lose things).

Lather, rinse, repeat until your game is perfect.

29 Responses to “Ron’s Rules for Playtesting”

  1. D. Moonfire Says:

    Those are pretty good suggestions for playtesting. I know that when I got CuteGod to a playable state, I watched my wife (who doesn’t game) play it and that points out a LOT of bugs that I needed to work on.

  2. Mortal Says:

    great article. I usually do the same when developing websites – have a couple friends look at the site and watch them navigate. I think you can use the same guidelines for anything in engineering generally – games or office apps – computers or real world devices. good luck with the game.

  3. Asrr Says:

    really nice article

    but … I slightly disagree with point 1 – you still can learn much from player who already has played level he is playing right now (about replayability, learning curve of game etc)

  4. Ron Carmel Says:

    Asrr, you have a good point. I still think there’s more to be learned from a first time player, but I agree that there are things that can only be learned from subsequent playtests with the same player. I would change the title for #1 to reflect that, but then I’d have to change the picture, and I kinda like it :)

  5. » Ron, 2dBoy extraordinaire, on playtesting… Says:

    […] […]

  6. 6 Reglas para el Playtesting, por 2D Boy | El Chigüire Literario Says:

    […] 2D Boy se sigue cocinando World of Goo. Ron, otro de los creadores de este juego, escribió 6 reglas para poder hacer playtesting efectivo. Nótese que no es testing para conseguir bugs, es testing para validar que el juego sea divertido. […]

  7. Patrick Says:

    If you’re doing a multiplayer game though, I think there’s a lot of value to be gained from having people play single-player to figure out the mechanics, then rouding them up for a multiplayer round.

  8. Petri Says:

    Great stuff Ron!

    I agree with everything written down here. This is the way I’ve been doing it, but I have one problem. It’s about the taking notes. I’ve noticed that if I take notes while someone is playing the game, the player gets very uncomfortable. It seems to bother them a lot that I’m taking notes while they’re playing. At some point it actually starts affecting how they played the game. They feel that they are being tested, instead of the game being tested.

    So I dropped the taking notes part while someone is playing the game. I’ll do right after the play testing session, but I can’t help feeling that I might have missed something, because I didn’t write it down immediately. Any suggestions on how to solve this?

  9. Ron Carmel Says:

    that’s interesting petri, i’ve never noticed players getting self-conscious about my note-taking, maybe i’m just oblivious :). perhaps if you tell them ahead of time that you’ll be taking notes and explain why they’ll feel more at ease?

  10. kotsoft Says:

    yeah, i have noticed that in my games, people usually have a LOT of trouble figuring out the puzzles i made that seemed easy for me. it’s a lot easier to make a puzzle than solve a puzzle i guess.

    and i agree with petri. i always get uncomfortable when somebody is looking at the monitor behind me.

  11. Steve Swink Says:

    @ Ron/Petri: We’ve had a lot of success videotaping playtests. I often just get people started and then just leave. Cameras are so ubiquitous these days (security cams in every convenience store/bank) that people tend to forget about them in a couple minutes and just act naturally, which is the goal of any playtest. We’ve gotten some awesome stuff from listening to people talk to themselves as they play.

    Another thing I do is tell people that the game they’re testing is something a friend of mine is working on (not mine.) This mitigates the “my baby” effect. If they don’t know better already, that is. If you don’t do that you always get softball lame-o feedback. People can’t help it; it’s subconscious, they don’t want to tell you something you’ve worked on sucks. Tell them it’s someone else’s work, though, and hooo boy. You’ll get a firehose of honest feedback that’s hard to turn off and hard on the ol’ ego :).

  12. Petri Says:

    Hey guys, thanks for the suggestions.

    Maybe it was just the first two testers, I’ll try the note taking again with better explaining of things to testers at the beginning.

    Also it could be the genre of the game. Because my game is puzzle game and people could take that as a challenge to their intelligence. And when they are struggling with a puzzle they probably see it as I’m somewhat questioning their intelligence. Me taking notes behind their back probably doesn’t make them feel any more comfortable about the idea. I could probably get over this with briefing them a little better.

    Thanks Steve for the video camera hint. I was already planning of setting up one of those see through mirrors in my student apartment :) The video camera is a great idea, I’ll have to get to habit of using it.

    Also I never trust what people say after playing my game.Their feedback isn’t really honest, because of the “my baby” effect. I have couple of good friends who know me well enough so they dare to give me honest feedback. But I’ve already over used them in the development of the game.

  13. Fun Link Friday » Games News and Reviews » Binary Joy Says:

    […] Boy have an interesting post on how they beta test their games. To tell the truth the process is pretty close to my method – although I suspect they are more […]

  14. pixelame.net Says:

    Reglas para testear un juego

    Ron Carmel, desarrollador independiente (2D Boy), nos cuenta algunos de sus trucos para testear juegos.

  15. Wrongfire Says:

    I really liked this article, i haven’t really made anything requiring playtesting yet except for one game, and I kept saying “You need to go here, and do this” Thanks to this little blog I know thats really not what I should do. If they are confused then I did something wrong =D

  16. Studio-Owens.com Says:

    Very nice blog post about play testing. It hits the major points of interest. Well done.

  17. Levi Watts Says:

    Thank you for this article. I have put people in front of my creations before, but never realized that my own input while they were playing affected the game.

  18. Foa Says:

    The rules are amazingly useful!!!

    Rule 1. is Very true
    Rule 2. is USEFUL
    Rule 3. is Keeps the first rule long-lasting
    Rule 4. is Helps the game and helps the fifth rule is fulfilled
    Rule 5. is the heavy-puller for the rules
    Rule 6. is Simply awesome! It is listing to the peoples needs.

  19. Chris Blow Says:

    These are great recommendations — I work for a research firm ( http://boltpeters.com ) that does gameplay research and these are right in line with what we do and what we preach. User testing is extremely valuable if you are, uh, actually interested in making something kick ass.

  20. George Warner Says:

    Camera’s are 100% the way to go here; A MacBookPro with a screen saver running will be ignored; iSight camera and all. Put it on a desk across from the testing location where it can capture as much of the game screen and the player as possible. A mirrors on the wall behind the game screen monitor can be used to get a better recording of the players face.

    A step up from that would be the one-way mirror with you and a camera behind it. Just make sure the camera captures the audio from both the play tester and you (this is much better than taking (written) notes).

    On the “dirty tricks” side, an “over-caffeinated” tester typically plays much more aggressively. Free coffee, soda (Jolt, etc.) and/or “energy drinks” can vastly improve the quality of your play testing data.

  21. MochiLand » Blog Archive » Promoting Your Game - Lessons Learned Says:

    […] Playtesting is what turns mediocre games into awesome games. Extensive playtesting is what made the award-winning game Portal great. If you want to learn how to do it right, read Ron’s Rules for Playtesting. […]

  22. MochiLand » Blog Archive » Why the Masses are “Right” Says:

    […] If you are going to make the most of out your friends and family, I highly recommend checking out this article by Ron Carmel of 2DBoy.com. It’s a quick read and has great tips for making the most out of the […]

  23. JARUAR (RUS) Says:

    CooL game !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

  24. Paul M. Says:

    Girlfriend’s parents got her a Wii for Christmas, so I began looking into gift ideas for the New Year (since they didn’t tell me they were getting a Wii, I was unprepared). After getting her Endless Ocean (she once had a dream of being a marine biologist, and she loves it), I looked into getting something for me. All over the net, I saw raving reviews of World of Goo, so I read descriptions. They had me at “simple yet addictive gameplay the likes we haven’t seen since Lemmings or Tetris.”

    She was skeptical (1) about the game itself and (2) about connecting her Wii to the internet. But, long story short, she was there when I played it for the first time. The music hooked us. The game itself addicted us. She may even be more addicted than I am. We have had it for five days, and it is easily the most fun I’ve had with a new game… possibly ever.

    In short, a request: will you release a World of Goo soundtrack? I know some of the tracks are just 20-second loops, but those can be expanded to 60-90 seconds or so, and the longer loops can go for a bit longer. But, really, it’s an incredible game and the music is absolutely beautiful — with or without the game attached.

    Thanks for the good times. I look forward to more from 2D Boy!

  25. axcho Says:

    Wow. I finally realized what you meant by “you also reach a higher level of enlightenment”. The first time I read it, I passed it off as a non sequitur. :)

    These are great guidelines. I link to them all the time. If you wouldn’t mind, I’d find it very helpful to hear of some examples where you ended up changing something in World of Goo based on these playtests. I’m really curious how World of Goo looked in its unpolished, unfinished state.

  26. 2D Boy: I love you, 2D Boy! » Blog Archive » the world (of goo) wasn’t built in a day — part 5 of 7 Says:

    […] skeleton of the game is now in place and at this point we started playtesting.  at first we wondered if we were showing the game to the right people.  why are they so bad at […]

  27. Hamid Says:

    thanks for sharing your experiences.

  28. Tien Hock Loh Says:

    Great sharing, I’ll see to using this as a guide to my next game :)
    Thanks a lot 2DBoy

  29. Tester une mécanique de jeu « Script à Part Says:

    […] Tester une mécanique de jeu By Anaël La méthodologie de 2D Boy (World of Goo): http://2dboy.com/2007/11/12/rons-rules-for-playtesting/  […]