IndustryGamers - Informing the Games Industry

Feature: Metacritic's Industry Impact

Posted September 27, 2010 by Anthony Neal

For gamers, Metacritic is as divisive a topic as they come. Fierce debates rage everyday among fanboys and girls about the superiority of any of the three current-gen consoles. I knew this when I did my study back in May comparing the Metascores of exclusive games for the Xbox 360 and PlayStation 3. It was a given that some would find it interesting, some would use it as ammunition in whatever platform argument they would have in the future, and some would write it off as a completely pointless exercise. However, seeing the level of discussion it generated here and at other gaming sites was intriguing. There was more to this topic of Metacritic that I hadn’t even touched yet. Not only has Metacritic changed the way that critics have to approach their dealings with publishers and developers, but it has also affected their approach to their websites, as well as the way that publishers’ public relations staff interacts with any given critic.

Even though it is a third-party that simply collects and averages critics‘ scores, publishers and developers are hesitant to discuss the subject of Metacritic. Several times I asked for comment from a publisher or developer, only to be either outright rejected or completely ignored. In fact, the only group of people that were always willing to discuss it were the people that are probably the most affected by it: game critics. As such an interwoven part of the industry, for better or worse, it is important to understand the impact that Metacritic has on the industry.

Metacritic is a double-edged sword. It provides a service to the consumer, but in the process, boils down a several hundred or thousand word review of any game to a simple numeric score. A reviewer often pours hours into playing through a game and then writing their review, but oftentimes, a consumer goes to Metacritic to grab a quick glance at a wide array of scores to save time. In one sense, this is completely understandable. “With the price of a AAA title going for north of $50, it proposes a significant financial risk to consumers, especially since no major retailers have a return policy if you are dissatisfied with your video game purchase,” Jesse Divnich, Vice President of Electronic Entertainment Design and Research, shared with me. “However, a consumer can go out and buy a miter saw from Home Depot without reading a single review, because there is little risk to the consumer. If it doesn't work to one's satisfaction they can simply return it. But again, this is not true for video games, once you buy it, you are stuck with it.” In the current economic climate, it is hard to argue with that logic; no one can blame consumers for doing research. What happens, though, when the very site that is trying to provide that service to the consumer base actually affects the business practices of the very people whose work they are relaying to the consumer?

Let’s Start at the Very Beginning

First off, the games industry is just that: an industry. It is, at its heart, a business, and businesses obviously exist to make money. The games industry, as Divnich pointed out above, is an industry that requires a significant financial commitment with each generation. A person can go see a matinee for $10 or $12, but the average new release game costs anywhere from $50 to $60. In reality, things somewhat equal out. If you can get 12 hours of gameplay out of any given $60 title, you’ve essentially paid $5/hour for that game. Granted, that doesn’t account for the cost of the system and accessories, but I’m just talking about the actual game here. A two-hour matinee that costs $10 also comes out to that figure. The difference, though, is that a person doesn’t usually see multiple movies at the theater in one day. A video game, on the other hand, is a one-time, out-of-pocket exchange of $60. As Divnich pointed out above, that’s a fairly significant investment. Because of that, he says, “aggregate review sites such as Metacritic and GameRankings have a significant weight in impacting the commercial success or failure of a video game.” Not everyone agrees with this notion, though. Chi Kong Lui, co-founder of GameCritics.com, points out that, “critically successful games usually indicate a degree of higher quality, but I don't think you could directly correlate those sales to any aggregate scores. You also have critically well-received games that don't do well in sales.”

Pressure to Make the Grade

The score a Metacritic or GameRankings provides the consumer is a snapshot of the perceived quality of the game by the community of reviewers and critics. If a game is perceived to be bad, there is the possibility that it will not sell well. It is not a guarantee that a poorly received game will not sell well, but it is a higher probability. Because of this, publishers are going to try various means to help their product succeed. In some cases, that’s pouring large sums of money into a marketing campaign. In other cases, it can be through direct contact with the reviewer through their public relations departments. Scott Jones, formerly of Crispy Gamer and now with Reviews on the Run, penned an article I referenced in my first Metacritic piece that detailed his personal experiences with pressure from publishers’ public relations departments to change the score that he gave their game. I contacted Jones for this piece to further discuss those instances, and while he didn’t wish to rehash those particular cases, he was more than willing to discuss this subject with me.

“I don't necessarily blame [Metacritic] for these incidents. What I blame is the trend towards the increased amount of pressure on PR and marketing [departments] to deliver high MC scores, as well as the overall dilution [of] the importance of the video game press in recent years,” he said. “Journalists are poor and vulnerable. Many of us are out of work these days, and more susceptible than ever to publisher/PR pressure. Remember: Publishers decide who gets review copies and who does not. So I think there's a trend towards PR/marketing feeling like they can strong arm writers and editors into doling out the review scores they need to justify their existence to publishers."

Lui, however, had a very different opinion on this issue. When I asked him if he had any knowledge of publishers pressuring their PR people in this vein, he replied that, “PR folks are under a lot of pressure to deliver positive press, but I don't think the problem is as rampant as some articles would have you believe. For every nightmare story that [you] hear about, I'm sure there are dozens upon dozens of professional interactions with PR people that aren't tabloid worthy.”

Jones’ case of a PR rep directly asking him to change a score he gave is definitely on the extreme end of the spectrum that PR departments have explored to try and secure a high score for their product. Oftentimes, I found that much more subtle methods have been used. Dan Hsu, co-founder of Bitmob.com and former editor-in-chief of Electronic Gaming Monthly, shared his personal experience on publishers that have refused to give him review copies of games because he would not guarantee a certain level of score. “[The] publishers that have done this sell it a lot better than that. They spin it more along the lines of: 'Hey, if you can give this a certain score, we will let you publish your review early' -- which gives the outlet additional traffic and thus, incentive to rate the game better. It may not outright bribe a lot of reviewers, but it does taint the process. If a critic is on the fence between an 8.5 and a 9.0 -- and the 9.0 is what the game publisher wants -- might that influence the final decision, however subtly?"

Jones also offered up similar experiences. “No one ever has outright said, 'You won't be getting a copy of this game because of the score you gave our previous game.' But have game copies mysteriously gotten 'lost in the mail'? That happens all the time.” He continued, “Probably around 40-percent of my interaction with most PR and marketing people involves subterfuge and misdirection. 'Oh, we must have sent that out to the wrong address. Whoops!' Or, 'It's not our fault that we don't have review copies yet; the publisher hasn't delivered them to our offices.' Or, 'We only had a limited number of copies at the time, and all of those copies had been spoken for.' Or, 'There was a big demand for the title, so all copies went to retail.'”

I asked Jones whether or not he had heard of cases where a publisher had put pressure on a developer or PR rep to get a certain score. “Of course I've heard about this stuff,” he replied. “Most of the time these situations involve advertising dollars that are committed to a website or magazine by a publisher, and the review scores that said publisher fully expects to receive in return. The gist: You want our many thousands of dollars? This game needs an 8.5 or higher from your reviewer.” All kinds of thoughts started going through my head after I read that sentence; they were immediately tempered, though, as Jones continued with his answer. “No one is giving 10s to dog-sh*t games that deserve 2.0s. But do reviews appear on websites and in magazines that are slightly inflated in relation to ad dollars? Sure, that happens. No one discusses it openly, but it's always part of the gossip circuit. I've never personally been part of such discussions. But I have friends and colleagues who certainly have been.”

Anthony Neal has been a gamer since he was 5... back when he didn't know there was such a thing as good games and bad games. He loves movies, good barbeque, Mega Man, and is a proud, lifelong resident of Nashville, TN.