Re. EN World staff reviewers and good/bad reviews

[Fiery James]

Psion's always been square with us.

I remember some of our first reviews - the d20 thing was pretty new, and there was a lot of "new blood" in the review pool. We had been so proud and accomplished so much that any negative thing mentioned in a review really stung - even when the overall review was positive!

Of course, that's par for the course. As you get more experience and get more products (and reviews) in the books, you tend to get a thicker skin, and learn another important lesson:

Different people have different needs, different likes, and different priorities. Of course I think that everything FDP puts out is a 5/5 (or 10 out of 10, as the case may be!). We never sit back and say "let's aim for a 3/5 on this one!" -- We give it 100%.

But, what may be absolutely perfect in my eyes is only a C+ in someone else's.

The good reviewers know how to judge the product, present their view, but couch everything so that those who's needs will be met by the product can judge for themselves. I think Psion's always been decent at this.

So, maybe it was a publisher who's got a little more thickness to develop on his skin - give it some time, it won't matter so much. After all, the real test is in the sales of a product -- if you're selling well, do you really care if Psion thinks it's a 3 instead of a 4? And, if you're not selling well -- maybe he's right!

- James

 

[DM_Jeff]

I have a similar stance to Morrus and Psion. I must admit that at DM's Haven I have been "cut off" by two different companies who didn't like minor problems I found with their product and not only don't I review their items anymore, I have a sour taste in my mouth even dealing with such people, and look down on their company as a result.

On the other hand, paranoia may have had a hand in whatever company wrote this...there are folks out there (a huge minority, but they are there) that write scathing reviews that those of us that actually read the product know those folks are writing falsely of a product...and it makes you wonder who hired THEM to write a scathing review.

It's a shame there's no reviewer's certification program but at the same time I'd like to think a reviewer's history should help speak for itself in regards to who really reads a product and who's just shooting off steam.

-Jeff Ibach
DM's Haven

 

[enrious]

I tend to ignore reviewers until they have a fair amount of reviews under their belt and only then if they have reviewed a few products I own so I can get a feel for their thoughts/feelings as compared to my own when I read the same product.

 

[Orcus]

I just saw this thread and wanted to post.

I give a nod to Morrus for appropriately not naming the publisher. The problem is, however, that it may lead consumers to speculate who the publisher was and arrive at an erroneous conclusion.

By circumstance of timing, Psion did a recent review of Maze1 at about the time this thread began. So I want to let everyone know that no one at Necromancer games sent any such email to Psion. So I hope that puts peoples mind to rest if they believed it was us.

As for reviews, I dont care if they are good or bad. Necromancer Games, as an example, makes a very niche product--stuff with 1st edition feel. We are not trying to appeal to all 3E gamers and know that we wont appeal to all reviewers either. So when we get a bad review it doesnt bother me much anymore. I will say that it used to at first. Now I dont even really read them.

In fact, I didnt know about Psion's review (who I admire, by the way) until someone told me about this thread by posting on the Necromancer Games message boards.

I like Psion's reviews. He does several things that I think are key for a good review:
1. he has done his homework
2. he admits his biases when they are present
3. he states the good with the bad
4. he isnt a gushing fanboy or a basher with an agenda
5. he has the experience to do a review

As far as cutting people off from reviews, I am not even a part of that process. All Necro products are sent out by SSS/WW pursuant to a review list. Frankly, I dont even know who gets all of them. I like it that way. I dont like the "free stuff for good reviews" approach. That is bogus. Plus, that way I am insulated from claims of trying to buy good reviews.

Morrus, Psion, keep up the good work!

Clark

 

[d20Dwarf]

It seems to me that if you are going to put out a product, then encourage reviews for that product, then you should be ready to accept what you get. I don't know anyone that gets regularly trashed by the staff reviewers here, and as for comparisons....one thing I've learned is that one man's pudding is another man's poison. Sometimes I find myself shaking my head at a review, but then I hear people agreeing and talking about how much fun they are having in their game with that particular product. I just can't imagine something like a review being offensive unless it was factually in error or an obvious personal attack.

 

[Orcus]

"It's a shame there's no reviewer's certification program but at the same time I'd like to think a reviewer's history should help speak for itself in regards to who really reads a product and who's just shooting off steam. "

Jeff-

Right on.

When I do look at reviews I look at the ones by people who I know have some experience. There should be a certification program, since reviews should not just be "customer comments". They should reflect a knowledge of the game, the mechanics, the history of the game, how it fits with past similar products, etc. And I dont mean history of favorable reviews. I mean a history of good reviews--fair, done by someone with knowledge, etc.

Clark

 

[Morrus]

When I do look at reviews I look at the ones by people who I know have some experience. There should be a certification program, since reviews should not just be "customer comments". They should reflect a knowledge of the game, the mechanics, the history of the game, how it fits with past similar products, etc. And I dont mean history of favorable reviews. I mean a history of good reviews--fair, done by someone with knowledge, etc.

Perhaps the publishers could band together and set something up? With enough backing, it could be regarded as "meaning something" within the d20 community.

 

[Orcus]

Morrus-

Great idea. Problem is, if publishers get involved people would say we were handpicking our reviewers.

I think the best thing would be for review sites to post some review guidelines. That would help immensely. Plus, I think the review sites should weed out some reviews that are obviously fan boy reviews or bashing reviews by haters. That doesnt mean pull every 1 or 5. You can tell when a review is by a fan boy or hater.

I know there will be some resistance to a review site reviewing its reviews (how was that for a sentence). But I have long said that there is a difference between a "review" and a "customer comment" (prehaps called a "fan review"). "Review" implies it is done by someone we should trust: like Roger Ebert. You may disagree with him, but you cant deny he has the experience and knowledge to do a "review."

Clark

 

[Mal Malenkirk]

*Discalimer* I Have no idea about the particular detail of this case *Disclaimer*

Reviews are in good measures opionions so you can't expect a reviewer to to write a raving review about a product he doesn't personnaly like.

But you can expect enough objectivity from him so that is able to differentiate from a product that he personnaly doesn't like but that has merits from a shoddy product that really stinks

Reviewer do have an impact on what I buy and I want them to keep in mind that they are reviewing a product for people who do not automatically share their taste.

As I said, I have no idea of the particular situation that prompted this thread. But I just wanted to share an experience that happened to me: Hitman 47.

Hitman is a computer game where you play the part of, obviously, a professional killer. Just the concept appealed to me a lot and all I needed to buy this game was an average review. But gamespot gave it 5.2/10 or something like that. This basically means that the game was deeply flawed in the reviewer opinion. Shovelware like Skydiving! usually gets these kinds of marks. Since Gamespot is usually a solid source of info and that I hadn't followed that game development, I decided I wasn't about to risk 60 bucks on such a game. I guess I could have shopped for different reviews, but the first time I had even heard of Hitman was from the gamespot review. Understandably I moved on and didn't think about the game for a while.

A year later I stumbled by chance on a 85+% review in a UK gaming site. I looked back on gamespot and realized that after a year the reader reviews (over a thousand) were at 8.6. I couldn't believe it. Then I bought the game and it turned out to be one of the most fun I have ever played! It's one of my favorite game ever and I almost missed on it because of a junk of a review. I could understand that someone would dislike Hitman on a personal level. It even had some controversial design decision such as no save feature during mission. But a review of 5.2 was unnacceptable, even if you account for personal dislike. The game had too many strong points that could be objectively noted.

The bottom line is that the reviewer should have been able to put himself in the shoes of different gamers and realize that while that game wasn't his cup of tea, it had plenty of merits.

 

[Lady Dragon]

I think to much negitivity is assotiated with a 3/5 reveiw.In my my 3=average certainly not outstanding but not bad either.I feel that five should be saved only for the best of the best.and 4's to obviously superior products.

If reveiews are done correctly I would say that a majority of all products should recieve 3's

1's should only be assigned to total garbage anyone giving a product a 1 I would have to suspect there motive or question what they expected to get out of the product.

2's mean below average They could have done better 3 as I said earlier is average.every product can't be innovative and inspiring.

One question I would ask is I wonder if there is any why to get some kind of bestselling list.I know that just because something sells or doesn't sell doesn't always mean excellent but gamers are intelligent people and I think they now whats a worthwhile purchase and what's not.