Game reviews generally suck. But, it’s entirely not our fault. Here’s why, and here’s why it’s important.
We do what we can. We write, we delete, we write some more, we tear at our hair, and we make a futile effort to shoehorn our experiences into what’s fundamentally a broken descriptive system. When you get right down to it, what does an “A” really mean? How comparable is Pathologic, with a score of 67 on Metacritic, to Rock Band, with a score of 92? Does the higher score mean one has greater objective worth than the other or that you should avoid Pathologic because “it’s a bad game”? Where does Wii Fit fit in, and what the hell do we do with Jam Sessions?
This rampant obsession with a single number as a representation of “worth” is probably the single most damning influence in experiential gaming today; it’s our personal albatross, willingly carried around our neck despite our knowing better. I’m as guilty as the next person of relying on Metacritic to guide my dollar spend, but I know that it’s not right. For every Braid that slips through the system by the skin of its teeth on sheer novelty value, there’s an enclave of amazing indie games that are supposedly not worth playing.
The Path got an aggregate score of 80. Stop and think on that one for a few seconds. And, once you’ve reflected on that, go out and buy the damn thing.
The biggest problem we, as writers, have is twofold: one one hand, we don’t have the maturity of interpretation afforded by history and, on the other, we’re dealing with a fundamentally different medium. But, that pales in comparison to the problem faced by gamers; the approach to critical analysis we’re used to reading marginalises the innovative, masks the creative, and reinforces the repetitive. Collectively, we’re the proverbial insane man, doing the same thing over and over again and desperately hoping for a different result.
The beauty of watching Margaret and David or reading Ebert is that almost everything they say is passionate, considered, and most importantly, subjective. They rarely discuss technical elements unless they’re substantially lacking or stunningly progressive. And yet, we as game reviewers still dwell on technical elements as if they’re the focus and, on some level, truly believe that we’re giving objective measures of worth every time we carve a score into hypertext.
On one level, we’re firmly to blame – we control the keyboard, we choose the words, and we make the final call. However, on a more important level, we just simply don’t have the confidence or language to discuss these fundamental differences between games. This will eventually come – it’s just a matter of time. And, with luminaries such as Michael Abbott and Mitch Krpata trying to attack critical review from new directions, we’re gradually getting there.
However, it’s an interesting contrast to compare the challenges associated with reviewing movies against those involved in reviewing games. Simplistically, it’s easy – both are typically narrative-based (French cinema and PopCap notwithstanding), both rely strongly on technical elements, and both primarily use audio/visual elements to communicate story elements. But, and this one’s a doozy, the difference is the level of interactivity offered by games.
Recognising that this difference exists is nothing new, but oh what an impact it has. Just as an otherwise unimpressive game can be transformed into a singular experience simply through spot-on controls, an otherwise emotionally moving game can be undermined by a sluggish, unresponsive control system. There just isn’t an analogy in any other medium which, in turn, begs the question: Why do we base our reviews on what are fundamentally unsuitable approaches?
And so, I propose a new metric. Paraphrasing Fight Club, I’d be naïve to believe that I’m a beautiful or unique snowflake – I’m painfully aware that I’m very much not the first. The thing is though, I’ve yet to see a holistic approach that’s actually useful and generalisable.
At its core, a useful review needs to balance:
- Efficiency, in communicating the intent in as few words as possible.
- Relevancy, in being perceived by the reader / viewer as being worthwhile in their personal context.
- Informativeness, in providing a breadth of multi-dimensional insight.
- Interestingness, in maintaining the attention of the reader / viewer.
These aren’t necessarily complementary; every step towards efficiency implies a step away from informativeness. Relevancy is almost totally contextual – what one audience considers relevant may be considered totally irrelevant by another audience. The role of the reviewer is to understand these conflicting pressures as well as their audience and craft something that will not only resonate but be remembered.
The key to the solution is that, for games at least, it’s impossible to do all this when all you’re relying on is a single number. Unlike movies, gaming is an active experience – the enjoyment appreciated through physical interaction with a game can be almost totally orthogonal to its narrative and technical delivery. MLB: 09, a game which I’m very much having a love / hate relationship with at the moment, has almost no narrative or artistic worth, but I’ll be damned if it isn’t a well-polished package when it comes to mechanics.
It deserves recognition for technical delivery, but trying to communicate that through a single number is not only pointless, it’s actively disingenuous - it misses the entire point of the game, not to mention forcing an implicit comparison against other “similarly scored” games. Ocarina of Time, Tony Hawk’s Pro Skater 2, and Halo all have similar scores and yet it’s painfully obvious that they’re totally incomparable.
So, possibly the first step to solving the problem is to recognise that maybe, just maybe, we don’t have to play by the same rules as everyone else. Maybe games really are different and maybe they really do require a different vocabulary and framework to critically assess.
Theories of play are, at a research level, quite well developed; there are entire games built around Skinner’s theories and entire languages built around understanding gamer imperatives. Game reviews, on the other hand, are about as dated as the Ford Model T in a world where the Prius is the market leader. For a review to be useful to the vast majority of gamers, at a minimum it must consider:
- Contextual Challenge (Difficulty): The difficulty level of the game compared to similar games in the genre.
- Technical Implementation (Quality): The contextual quality of technical elements such as graphics and sound.
- Experiential Uniqueness (Art): The subjective “worth” of the experience.
- Time to Mastery (Longevity): The length of time required to truly master the game and the mechanics.
- Depth and Complexity (Depth): The level of strategic and tactical depth offered.
- Personal Interaction (Multiplayer): The degree to which the game allows player-based emergent interactive gameplay.
- Personal Enjoyment (Fun): The subjective level of enjoyment appreciated by the reviewer.
These aren’t arbitrary; I’ve deliberately selected them because they align to certain behaviours, interests, and gamer psychologies. And, the reality is that if most games were reviewed on a more multi-dimensional scale similar to the above, the vast majority of them would inevitably be found lacking. Assassin’s Creed, one of the major releases of last year, would score extremely poorly on five of the six dimensions above. Bioshock, one of the more highly acclaimed games of last year, would probably score well on only two, maybe three of the dimensions and poorly on the others.
Over the next few weeks I’ll investigate this more thoroughly as well as provide practical examples; if we’re to outgrow our sophomoric obsession with marines carrying big guns, we need to develop a deep enough language to discredit them.
Not to denigrate Mitch and Michael but I think that the work undertaken by RPS in their “Wot I think” articles, and the old Games For Windows Live podcasting crew did more to advance games reviews beyond a single score. RPS actively write about games passionately from a personal perspective but one that also acknowledges exactly where they’re coming from, collectively and as individuals, and with ridiculous clarity. They also write “Wot I think” in a way that indicates to the player what they’ll undertake when playing a game. Michael and Mitch often highlight areas that they feel should be addressed, but more frequently they raise hard to answer questions, which are necessary but far less informative.
Perhaps Abbott and Krpata are more useful to an audience that wants to understand games, like game designers and game writers, and RPS are more useful to an audience that wants to play games.
Reflecting on the multiple categories that you say must be covered by any decent review, plus this line:
“It deserves recognition for technical delivery, but trying to communicate that through a single number is not only pointless, it’s actively disingenuous – it misses the entire point of the game, not to mention forcing an implicit comparison against other “similarly scored” games.”
It almost sounds like you want a return to the days when reviews gave a score for graphics, sound, gameplay, longevity and fun. I know that’s not what you really mean, but it is worth noting that this style of reviewing is so passé these days (although not extinct) and at the same time the single-number review that is so prevalent is considered too simplistic. In an ideal world, of course, everyone would read the text of a review, ignoring any number score, and come away enlightened, but you you said it yourself: even those of us who think that way use Metacritic.
By the way, is it intentional that the author’s name on each article is hidden in small font down by the tags?
@Daniel: Absolutely – “Reviewing the reviewers” is probably worth research in its own right! Picking Abbott and Krpata specifically was probably more an indication of what I’ve read recently than any intentional subtext; Mitch was down on Brütal Legend and Michael just posted a video of his son playing the drums on Rock Band, something I can appreciate given my daughters about the same age.
@Fraser: I think you’re on to something there – there’s a distinction between critical analysis and critical review, even though they often overlap.
The last thing we need is a move back to scoring every single technical element individually; personally, I’d lump all of those under a single “Technical Implementation / Quality” metric and be done with them. Having said that, I still played Assassin’s Creed to the end, even though it was possibly one of the most boring games I played last year.
My logic behind the dimensions I’ve chosen relates back to Caillois and a lot of what Bateman’s been talking about over the years over at Only a Game – we really do play games for different reasons. Some of these are actually implicitly communicated by the types of sites that run the reviews, but as general model, there pretty much always needs to be some level of coverage across all these elements for a review to actually be comprehensive and useful. The Gamespy audience probably isn’t going to care so much about Experiential Uniqueness, but again, that’s implicitly communicated if you’re a regular Gamespy reader – you don’t expect it, you don’t care about it, and you don’t want it.
We still get the information we need, we just typically get it from 20 different sites because not a single one of them normally manages to accurately describe the game completely. I’ll bet you $10 that even when you use Metacritic, you still scan for three or four sites that align to your interests based on the seven elements I’ve described. And, after finding them, you’ll weight their opinions greater than every other score there.
@Fraser – Yeah, the hidden byline is to mask the fact that Evan is far more prolific than the rest of us
One of the ideas for an article that I was tossing about was to go to see a film and write a review on it as a game reviewer would. I imagine the outcome would be very different from that of your average film review. A combination of a general fear of the outcome, alongside a 12 week old daughter, is keeping me from going through with it though.
While I’m aware that you’re not stating that we should be wandering around a game kicking its metaphorical tires and testing to see how fast it goes from zero to 100 etc. It does sound as if you’re advocating some sort of clear framework for reviews which to me is quite a dangerous proposition. If followed to the letter of the law, any set of guidelines decrease our ability to provide useful insight and limit us in viewing these creations on their own merit. However, I do quite like the idea that you’re pushing of moving beyond the technical aspects of a game in order to focus more on other points of differentiation. You’ve got some good points to ruminate on.
Can’t wait to see where you go with this.
I’m definitely not recommending a prescriptive, paint-by-numbers reviewing structure. What I am saying, though, is that reviews should broadly match our actual play and thought patterns (and leverage the research that’s already been done into gamer psychology), not just be an arbitrary brain dump of whatever pops into one’s head.
What I’m advocating is a framework for thoughtful consideration, one that covers substantially more ground than the current norm and one that’s better aligned to the medium in which games operate. It doesn’t replace the need for narrative analysis and deconstruction or anything else.
I think (good) film reviews have moved to a place where we’re comfortable with a combination of product review and criticism. They’re both reccomendation and discussion, they deal with both enjoyment and meaning. They’re also unpretentious, as not every review of a top film is seen as ‘important’ or ‘a work of art’ in and of of itself, and as such higher scores (five stars etc) can be given for things that are simply ‘great’, rather then having to be Citizen Kane.
I always try to do this with my reviews of good games, I want to slip in some of the ‘meaning’ inside the product review side of it. I want people to understand a game from a review I do, rather then just want to buy it. I’m very proud of my reviews of Ouendan 2 and Zelda Phantom Hourglass (among others) for this reason.