About Playtesting

by Malte Skarupke

So I’ve been thinking a lot about playtesting recently. Mainly triggered by Portal 2, my experience in making the game Leshy over the last semester, and also by lots of other stuff like this talk by Jonathan Blow.

I was showing the finished version of Leshy to our composer, Tyler and I was watching him play the game. He eventually got to an area where he got stuck. And I knew exactly that the reason why he was stuck was, that he had gotten to this area too early and that he didn’t yet know about a game mechanic that another area would have instructed him about.
If you have ever done a playtest, you know that this is a horrible situation. In playtesting you are not supposed to intervene. You don’t give hints, you just watch the player play and see if he figures it out or not. So I was sitting there watching this guy trying various different things, all of which were destined to fail. It was extremely frustrating to watch this. I knew that he couldn’t succeed because the game hadn’t instructed him yet, but yet he kept on trying. He got it almost right and failed. He got it very wrong and failed. He tried the very wrong thing again and failed. He failed, failed and failed again. And I’m sitting there just wanting to shout and scream and just tell him how to do this, and that this wasn’t his fault and that we’d fix this and oh why won’t you at least give up and go somewhere else first so that you may come across the training section that you missed.

Get the sphere up to those stairs. Shouldn't be that hard, right?

Anyway. That was my impression. He on the other hand was having a good time. For him this was a challenging puzzle that he was trying to figure out. It is totally OK for a player to be working on a puzzle for five minutes, at least as long as he can keep on trying new stuff and isn’t completely stuck. And he did eventually figure it out.
I don’t think he was frustrated at all. It was just me being frustrated. Below the jump I think about Valve in relationship to this experience:

The team that made the game Tag, who got hired by Valve to work on Portal 2, came to talk at DigiPen recently. They mentioned the importance of watching people playtest the game, because then you “feel their frustration.” I’d agree that you have to watch people playtest your game, but I think you shouldn’t mistake your frustration for the player’s frustration.

Now Portal 2 is a game where playtesting was core to the development process. And I think as a result they made the game too frictionless.
Here is a screenshot from about two hours into the game:

Test Chamber 17

What I would like to point out is, that there are almost no surfaces that you can shoot portals on. There is the area up high on the wall, and the three small portal-allowing surfaces lower. This is a very typical layout of a puzzle room in Portal 2: If you don’t need to shoot a portal on a surface, you can’t shoot a portal on that surface.

This is a puzzle that you can solve through thinking. I solved this puzzle by stumbling across the solution, because there simply aren’t many possibilities. I was still exploring the room, figuring out the possibilities that I could work with before I’d start thinking about how to solve this, when I had suddenly solved the puzzle.
This is a puzzle that is far enough into the game, where in Portal 1 you would have encountered really difficult puzzles. Here they treat me like I am an idiot.

My theory is that Valve had many moments like I experienced in my playtesting session above: They watched a player fail over and over again so they changed the level:
They watched a player shoot a portal over an acid pit, step through that portal and die. So they reasoned “what is the purpose of having that portalable surface over the acid pit? We wouldn’t lose anything by removing it.”
Then they watched a player shoot a portal that allowed him to walk under some stairs that he wasn’t supposed to go under, and the player was disappointed that there was nothing there for him to discover. So they changed that surface so that players wouldn’t be disappointed and would know instantly that there is nothing to discover under there.
And they kept on changing the game, removing all the small and big frustrations, until most people could solve most puzzles within five to ten minutes.
Which is of course not very interesting, and kind of defeats the purpose of the game.

If you haven’t played the game: The fact that there are almost no portalable surfaces in the game doesn’t change until the very end. For the vast majority of the game, you can not shoot portals on more than 90% of the surfaces. There is also a new obstacle in Portal 2 which hasn’t been in any previous Valve game: The waist high railing. It is insurmountable in Portal 2. This obstacle is best known from corridor shooters where you aren’t supposed to go off the path. It being in Portal 2 is just one of those small things that tell me that Valve took some steps in the wrong direction from Portal 1.

Test Chamber 08 (Portal 1)

Look at this screenshot from the first Portal: You can shoot portals on almost every surface. The only exception are those close to the exit, because you could probably find a way to cheat your way through the room otherwise. Also: You can jump over that railing into the acid if you wish, which is impossible in Portal 2. Also: That energy sphere kills you. Nothing in Portal 2 kills you. I don’t think anyone complained about these things in Portal 1.

I had the misfortune of noticing very early in Portal 2 that the entire experience is much more controlled. And then the game just isn’t very immersive. It suddenly becomes obvious that you are just walking down a corridor. Which made sense in Portal 1, but doesn’t in Portal 2. There are multiple scenes where you escape the test chambers. And then you are in a huge area, but the game quickly disappoints because you can not do anything. This isn’t a huge area where you can try to escape the compound using portals or otherwise show off how clever you are. It’s a very big-looking corridor. Just keep walking down the corridor and you will quickly be out of there and in the next test chambers. If you come across a surface that you can shoot a portal on, then you need to use that surface to continue along the corridor. You can never be inventive in these big areas.
When you notice the corridor, the entire thing doesn’t make any sense to you. The character and the player have no motivation to walk that way. You simply walk that way because that’s the only way you can walk. You get high up on a platform not because the character wants to, but simply because that’s where the only surface that accepts portals is.

Portal 2 intro: You get glimpses of wide open environments. If at this point you start imagining the potential of the portal gun in these environments, you will be disappointed: When you do get to environments like these the gun is pretty much disabled.

I’d argue that it was important for the enjoyment of Portal 1 that you do mess up in stupid ways. Because when the designers gave you freedom to mess around and try crazy stuff, you also had the freedom to solve the puzzle in unexpected ways. Or at least it felt that way. In Portal 2 every solution is obviously designed to be a solution.

I don’t think that it was a conscious decision on Valve’s part to limit the player so much. They probably never said “alright, let’s remove almost all surfaces that you can shoot portals on.” They probably got that from playtesting and iterating and they probably didn’t even notice the game changing over time. They don’t let the player die in unexpected ways and they don’t let the player lose his direction and they don’t let the player spend minutes on trying stuff that’s not going to work and they remove every negative aspect of the portal gun.
And nobody is complaining about this because playtesters don’t complain about this kind of thing.
And then you’ve made a game where there is a confused “this is brilliant but not satisfying” kind of reaction and where nobody would say that they explored the potential that the portal gun would have had in these environments.