So why don't reviews work? (as a marketing tool)

[Pinotage]

Lack of reviewer credibility? I know John Cooper has always impressed me, especially with his technical knowledge of d20. But for the most part, I don't think most others (including me) stand out much. (Not to say bad, but simply mostly generic). I think Monte Cook has also remarked on the unremarkable quality of review writing.

This is quite an interesting one which leads to another question for us reviewers. How do you improve your credibility?

Pinotage

 

[kensanata]

My purchasing decisions are nearly always backed up by reviews. The only products I buy unseen are my Paizo subscriptions. I've come to realize that reviews don't always tell the whole story, but reading them and taking that into account still seems like a better deal than being totally uninformed.

How do you improve your credibility?

I think the most important criteria for me are the following:

• Visibility. If I see that most of the products I'm interested in have a review by either John Cooper, Crothian, trancejeremy, Psion, or JoeGKushner, then I'll be interested in what they say next time I see them. If there are a dozen reviews, I'll only read the reviews by the two or three reviewers whose name I recognize.
• Explicit prefereces. The more I know about reviewers, the more I can take their own preferences into account. Examples are stat block errors, starting villages, artwork. Knowing these preferences, I can ignore (or pay extra attention to) that part of a review. That's great. It's part of the "signature" of a reviewer, and it gives me confidence in their review.
• Playtest review. This is the hardest. But if somebody says he played through the first book of Echoes of Heaven or the Drow Wars and liked it, then that beats whatever anybody else says.
• Personal notes. If the review author says that he will use X and Y in his next campaign, or his players didn't like options X and Y, then that is better than a summary on every chapter. The subjective opinions beat straight material previews.

 

[amethal]

I would love to get more reviews of the War of the Burning Sky adventures. Particularly because with the drop-off in sales since the 4e announcement, every little bit helps. I want people to know how good these adventures we've put out are.

So hell, would anyone like review copies of the adventures?

I'm about a third of the way through a review of the Fire Forest. Its amazing how little time I seem to have these days.

My aim is to review them all, after we've played through them, but this means the reviews will no doubt be far too late to impact on sales.

Reviews had nothing to do with me buying the WotBS subscription in the first place, however. That was a combination of a very competitive introductory price and a weak dollar.

 

[Li Shenron]

Utterly complex topic

Since a long time I got the habit of never buying a RPG book before having read all the reviews from ENWorld at least, possible more from other websites.

For me they are very important, but I believe that I'm in the minority, and I agree with the publishers who believe that reviews in RPGing are not useful. Most people buy stuff out of gamer's compulsion: they see the cover on the shelf or WotC website, or they read half a dozen posters in the same thread deeply detailing their personal opinion with a summarizing "W00t!", and they're sold. An advertisement well-placed and well-designed scores much more than a review on the average...

Lack of reviewer credibility? I know John Cooper has always impressed me, especially with his technical knowledge of d20. But for the most part, I don't think most others (including me) stand out much. (Not to say bad, but simply mostly generic). I think Monte Cook has also remarked on the unremarkable quality of review writing.

Review quality is subjective.

Personally I wrote only 3 reviews, spent a few evenings on each of them trying to be the best job I could. I am sure no one bought a copy of those books because of my review, and I doubt they have even been read by more than a couple of people... The fact is, I wrote that review based on what I myself would want to read in a review, but is this really useful for others? For instance, you mention John Cooper: his reviews are excellent, but they also serve a very specific purpose. Many gamers may not need that information at all.

Certainly, many reviews which are 5 lines long are seen as "useless" by many, since it's really impossible in 5 lines to list all the good and the bad of a book, with a little why. However, to many readers what sticks to their mind is just the final grade. If you have a low average, people may not even spend time reading the reviews. If you have 100 votes with a 99% average score, you can be sure that people will notice it!

In this regard, I think ENWorld has a problem, because you cannot "vote" a book unless you write a review, but writing a decent review takes time that few people accept to spend. The result is that most books have too few reviews and too few votes.

Just the nature of RPG reviews? I mean, unlike game reviews, a lot of RPG reviews are done just by reading the book, not playing it. But that often simply can't be helped. With sourcbooks you'd have to run a campaign that allowed you to simply drop in new classes/monsters/rules at a moments notice. And adventures can take weeks or months to run.

This is the greatest problems IMHO. RPG books are supposed to be USED not just READ. How can you truly be informative if you write a review BEFORE playing with the material? To review a PC game they sit down for hours in a bunch of people and play the game until they have a decent grasp on it, because you cannot judge a game when you still haven't understood the controls for instance... Instead, we write RPG books reviews and sometimes we haven't really read the whole book front-end. Playtesting reviews are the most valid ones, but how many are around?

 

[McBard]

If I'm at a store, I read the module.

This is my primary reason for not reading RPG reviews: I don't need them. My FLGS displays nearly all their books without shrinkwrap. In about 5 minutes of thumbing through any given book, I can determine whether I will buy it. (And if it there is shrinkwrap, the product's homepage will more often than not post a sufficient amount of PDF previews to inform my opinion).

My second reason for not needing RPG reviews: I'm as much of an expert on the topic as the "guy on the internet" (as someone put it aptly, above) who posts a review. I can't say as much for other products I consider buying (computers, cars, toaster ovens, et al), and so I do read their reviews.

 

[Kaladhan]

Playtesting schmaytesting.

What I'm getting from reviews is what the book is about. What it covers well and what it doesn't cover too well. Let's take Dark Ages : Europe. I want to know which countries are well covered and which are barely detailed. I want to know if the description screams plothook and inspires Storyteller.

In a perfect world, every book would be playtested before a review. But come on, you can't expect somebody to run a 1 year campaign and tell you how new feats are worth at every level of the game. Experienced DM can get a feel of the usefulness of something by reading the description and imagining what his player would do with it.

So I read review to understand what the book is about. I don't care much about the opinion of the reviewer because reviews are so subjective. The RPG hobby is very diverse. Even in Greyhawk, there's sub-division of those who prefered Gygax era and others that prefered 2E Greyhawk. Even with a new core setting of Greyhawk, some Greyhawk fans won't be happy with it. So my point is that I have taste very different from mainstream RPG. So I just want a summary of the product. Few or no opinion please.

 

[jaerdaph]

Would you care to elaborate on this, please? I'll admit that there are a number of reviews there, probably a fair majority, that are not very useful or informative, but there are an equally large number of reviews there that are good and informative, so I'm interested in where you're coming from? Do you think that being able to leave 'comments' on products rather than 'reviews' is a bad idea?

Yeah, there are some good comments at RPGNow, some even border on being an actual review, but lately people aren't saying anything substantial or of real value. I attribute this to over-zealous/over-eager (because nobody is leaving them any comments or reviews) to outright unscrupulous (because the few legitimate comments they get sum up their product as a piece of crap) publishers leaving their own comments or having friends leave them, and publishers offering coupons and discounts to anyone who writes a review or comment (not that there is anything shady about doing that, but the quality of what folks are saying in exchange isn't exactly stellar in most cases). Because of this, the 1 to 5 star rating system has become meaningless to me.

If I know someone and respect their opinion (staff reviewers and EN World regulars for example), I'll read their reviews or comments and they will hold weight with me. But ultimately, it is message board buzz by folks I know discussing a product (with publisher/designer interaction always a plus) that gets me to buy something.

Also see my comments in this recent thread:

General - "Glowing" review from a product not available yet?

 

[jaerdaph]

[threadjack]Schadenfreude - I recently read somewhere that this was named the Pseudo-Intellectual Word of the Year, but I can't remember where. I thought it might have been The Onion but I can't find the quote. Anyone know? [/threadjack]

 

[Arnwyn]

But there are others:

One generally doesn't want the customer to have to go and seek out your marketing materials. Nor do you want said marketing materials to be of questionable quality - the company does not control the quality of writing or accuracy of the content.

Reviews also tend to point out weaknesses as well as strengths. Nobody doing marketing ever wants to mention weaknesses of their own product.

This makes not one lick of sense, as it flies in the face of... well... all reviews (eg. films, video games, TV).

Could you be more specific/focused as to why the above would only apply to the RPG industry, and no where else? (I'd guess and say market size and reviewer quality mixed with timing, but it's your quote, so I'll let you explain what you're talking about.)

 

[Pinotage]

Yeah, there are some good comments at RPGNow, some even border on being an actual review, but lately people aren't saying anything substantial or of real value. I attribute this to over-zealous/over-eager (because nobody is leaving them any comments or reviews) to outright unscrupulous (because the few legitimate comments they get sum up their product as a piece of crap) publishers leaving their own comments or having friends leave them, and publishers offering coupons and discounts to anyone who writes a review or comment (not that there is anything shady about doing that, but the quality of what folks are saying in exchange isn't exactly stellar in most cases). Because of this, the 1 to 5 star rating system has become meaningless to me.

Also see my comments in this recent thread:

General - "Glowing" review from a product not available yet?

Thanks!

I'll certainly agree with you on the unscrrpulous part. I've seen my fair share of that in the past, but I'm not sure there's a lot that can be done about that. Publishers can milk the system quite easily (on RPGNow and Enworld, though not so much on RPG.net), and it's ultimately up to the readers to discern what's what, and what's not.

To be fair, as well, the system at RPGNow is currently written so that only people who have purchased a product can 'review it'. Publisher coupons are hence generally sent not to reviewers but to those who have purchased products.

Here's another question, though: How would you improve it? Ban reviews shorter than a certain length like here on ENWorld? Get staff reviewers to verify each review written?

Pinotage