IndustryGamers - Informing the Games Industry

Mass Effect 2 Metrics Measure Millions

Posted September 8, 2010 by M.H. Williams

Unbeknowst to gamers Bioware’s best-selling sequel, Mass Effect 2, silently collects player data through the game’s Cerebus Network. The information is completely anonymous, so Bioware has no clue that John Smith’s Commander Shepard heartlessly sacrificed a crew member for a better gun. The company’s Mass Effect Executive Producer, Casey Hudson, sat down with IGN to talk about the data and how it’s helping Bioware evolve the franchise.

"The only data that we get are in terms of events -- little things that happen in the game," said Hudson. "Let's say if we want to know whether players skip lines of dialogue, we can have that become a little event that gets sent up. It's all completely anonymous, so all we get is raw numbers for how many times these kinds of events occur. Then we can start getting ratios and comparing proportions and things like that. It becomes this mass of numbers, and then we have to try to figure out how we would interpret that."

"Sometimes you'll design something and think that it's going to be used in a certain way and people will use it in a completely different way. And if you didn't know that, then you would just keep making that system the same as you did before. But once you know what players like and what they don't like, based on the way that they're playing it, then you can make more of the good stuff and less of the stuff they weren't interested in,” he continued.

The data is being used for a grander design scale, so player should not expect the data collected to effect DLC, or the upcoming PS3 version. These type of metrics have been enjoyed by competitive FPS and MMO designers for years, but it’s a new trend for single player titles.

“Ultimately it doesn't always give you the answers, but it sometimes raises questions or gets you to ask the right questions…More people played the soldier class than all of the other classes combined. If you know that, then you can start thinking about future games. Is that good? Is that a problem,” he said. "It helps us to confirm a lot of the assumptions that we make when we design games. Obviously with a game like Mass Effect we're trying to capture the sense of continuity and the cinematic experience.”

Among the data, Bioware found that only 15% of conversations were completely skipped in Mass Effect 2. The rate grew during non-critical cutscenes, and dropped as the game reached the climax. "If we found that 80% or 90% of the lines were being skipped, we would have to reevaluate the work that we were putting into the digital acting."

The save import system, which was a big question mark for the developer, was proved to be a useful feature by the metrics. "We were surprised by how many people imported a game from Mass Effect 1…We put a lot into that feature and we could have gotten data back that said nobody was doing it. But actually more than half of players imported their save game from Mass Effect 1,” said Hudson.

The differences between Xbox 360 and PC players were not a pronounced as one would think. The average 360 time to complete the game was 33 hours, with PC players taking one hour more. “One difference was the people who did certain loyalty missions on the Xbox versus the PC, which is kind of surprising. On the PC for example, people did Miranda's loyalty mission quite a bit, which is where she is trying to connect with her sister and it's more of a touchy-feely plot. Not a lot of Xbox 360 players did that one. But the Xbox 360 players did do Grunt's mission a lot more than PC players."

For the hardcore, two players on the PC version played through the game 28 times, while four Xbox 360 players only reached a mere 23 times.

Bioware hopes that he data can be used to improve the player’s experience for the next title and beyond. "Ultimately what it means for players is that we don't have to guess anymore about what players are doing. We can actually learn about what they like and what they don't like and just try to focus on building the good stuff."

M.H. Williams has been writing in some form or another for ten years and has been a hardcore gamer since the NES first graced American shores.  You can catch him on Twitter as @AutomaticZen, Google+ as himself, or on his personal Facebook page.