Re. EN World staff reviewers and good/bad reviews

[Morrus]

I recently received an email from a publisher complaining that one of the EN World staff reviewers didn't give them good enough reviews. That publisher compared themselves directly to a specific, "obviously inferior" product line produced by a rival publisher (whom they claimed that the reviewer favoured) and made thinly veiled threats regarding "revisions" to their review policy.

I won't go into the whys and wherefores regarding what I think of this. I'm sure you can guess. Personally, I choose not to review or overly promote any products from that company again except as far as my "news reporting" role requires me to; I leave it up to the reviewer in question as to whether he wishes to.

As for the veiled threats - I'm sorry, but they're no threat at all. I know that I, and the staff reviewers here, have more d20 products than we could ever use, and are only personally interested in a tiny percentage of them (that does not mean that only a tiny percentage are good); if we really want a product, we can buy it. The concept of "free product for good reviews" really bothers me. If it's a major release, and requires coverage, then I'll even buy it for the staff reviewers myself.

For the majority of print publishers, a review here doesn't exactly make or break the company in question. A publisher is not going to go under because Alan or Simon gave a product a '3' instead of a '4'.

There, that's off my chest. There are a lot of great publishers out there who take the bad (or, more often than not, merely average) reviews in stride; that, in my opinion, is the professional way of acting.

 

[Lady Dragon]

Reviews are opinions nothing more.You are free to agree or diagree with them. Any publisher who can't stand the heat should get out of the fire.If his products are good the D20 gamers will notice,If there not maybe he should attempt to improve his products or get out of the buisness.

 

[Nikchick]

Heh heh. We're back to the "give 'em something for free and they'll complain the portions were too small" anectdote again, aren't we?

The only time I question reviews is when they're just blatantly misinformed, such as getting the price or contents wrong, or someone who clearly doesn't understand the genre ("There are no giant robots in this D&D book! Everything's better with giant robots! This sucks!"). I'll admit I raise my eyebrows at the people who review in order to rant about how they hate some particular format (mini-adventures, GM screens, whatever) and then have nothing to say about the product at hand except "Yeah, this is one of those! I hate those!"

It's a funny world, fandom.

Nicole

 

[MEG Hal]

I am of the opinion that as long as a reviewer shows that they looked at the material I am ok with it...it is when I read something that makes no sense that irks me a bit, but overall good or bad I have felt most really looked at the product anf therefore I was satisfied....but that is me.

 

[SSS-Druid]

An interesting phenomenon that I've noticed, however - on a well-frequented board, a bad review can often tell more about a product than a good one.

See, a miserably written review, while useless in and of itself, often generates a small outcry from folks, who are eager to correct the mistaken assertions made in the review. While you don't often see it on this board, over on RPG.net, a bad review often results in a huge exchange about the genuine merits and flaws of a book from wide variety of people.

Sometimes, you even get as many as three "retaliation" reviews, written so as to correct the perceived mistakes in the initial one.

 

[Mystic Eye]

I am of the opinion that as long as a reviewer shows that they looked at the material I am ok with it...it is when I read something that makes no sense that irks me a bit, but overall good or bad I have felt most really looked at the product anf therefore I was satisfied....but that is me.

I agree with Hal on this. As objective as a review might be there are always some level of personal feelings so we will all get good and bad reviews at some point. It is a problem when a product is reviewed on a cursory glance or overview only. The good thing is, the lack of homework often shows through on those types of reviews.

Ooops, I forgot to add that I think the staff reviewers here do a fine job at it.

 

[trancejeremy]

Well, if you look as some of the things movie companies do, this isn't surprising.

I mean, Sony got caught inventing a reviewer (they made the mistake of picking a real newspaper).

Most companies who have a dog of a movie don't even show it to critics.

Or the PC Games/Magazine industry.

Many times a game will get an absolutely glowing 6 page preview that makes it seem like it's wonderful. And thanks to the lead in time, this often is a month or two before the game comes out. But the game is released, is a dog, and 3 months afterwards, the magazine gives a 1 column review on it.

Still, this was a pretty silly accusation. Mr. Kohler's been reviewing stuff for years (on RPG Net before this), and he's been on the internet for a long time and is generally well respected. And while I've noticed that I often tend to disagree with Mr. Collins' opinions, he always gives good solid reasons for why he feels a product is bad or good, and is pretty thorough. I don't see how anyone could doubt either's honesty, or say they are biased.

 

[EricNoah]

I find it maddening that the supposed professionals in this case aren't acting anything like it. Which maybe begs the question -- how should a professional deal with negative reviews? I would hope that they would a) read them and learn from them, b) try to get a lot of reviewers so that a few negative ones won't skew things too much, c) put out a better product next time.

If you've put out a great product, it will show even over a few negative reviews. If you've put out a mediocre product, whining about negative reviews sounds more than a little desperate.

 

[HyrumOWC]

I'll never understand this kind of behavior either. Having been the "target" of more than a few negative reviews, I know that getting them does piss me off, but it also gets me to hunker down and do all I can do to make the next product even better.

And NikChik is right. Blatently false reviews just piss me off.

 

[Lizard]

Heh heh. We're back to the "give 'em something for free and they'll complain the portions were too small" anectdote again, aren't we?

The only time I question reviews is when they're just blatantly misinformed, such as getting the price or contents wrong, or someone who clearly doesn't understand the genre ("There are no giant robots in this D&D book! Everything's better with giant robots! This sucks!"). I'll admit I raise my eyebrows at the people who review in order to rant about how they hate some particular format (mini-adventures, GM screens, whatever) and then have nothing to say about the product at hand except "Yeah, this is one of those! I hate those!"

It's a funny world, fandom.

Nicole

There was a review recently of a product I worked on, which had otherwise garnered fairly positive reviews, which gave it a low score. The very brief review was, roughly, "I wanted this book to be about X. Instead, it was about Y. Thus, it sucks."

Now, if the back cover and ad copy had been "This is the ultimate guide to X! Fans of X, you'll be drooling! No lover of X can be without this book!", the reviewer would have a point. However, the content of the book was pretty much exactly what it had been described as in ad copy, and there had been quite a few sneak peeks of the contents prior to publication.

So it goes.

(On the other side of the coin, I recently read a review for a supplement to a game, by a writer who admitted he didn't know anything about the game the supplement was for. Now, think a second. If you don't know anything about D&D, on what basis could you judge, say, "Sword&Fist"? How could you know if it was balanced or unbalanced, if it had original ideas or just rehashed material from the core books, if the 'tone' of the book fit the tone of the core game, etc, etc, etc? Yet, someone decided to review a supplement for a game they didn't know, presumably just to post a review.)