Reviews

[Sir Trent]

I was just curious. I've looking over some of the posted reviews and am surprised by the high ratings seemingly given to every product. The few times someone gave a low rating to a product, he was flamed unmercifully. Why bother with the lower ratings if nothing is ever going to drop below a 3? It just seems odd that every product reviewed should be average or above. Am I missing something?

 

[Blacksway]

There are lots of theories about this but basically it boils down to the fact that more people buy products they know they are going to like, or that have already had good reviews and hence there are more good reviews than bad.

 

[malladin]

I also think the score system is too restrictive.

As an old Vampire player, I've grown up with the concept that in a 5 star rating, 1 is poor, 2 is average, 3 is good, 4 excellent and 5 mindblowing. The system here puts 3 as average, thus giving you much less leeway when determining how good a good product is.

If a reveiwer likes a product, but doesn't think it's amazing, they have to give it 4 stars, instead of choosing between 3 or 4. 5 is also over used because of this - a very good product will get 5 stars when it probably should get 4.

Hence you rarely see 1s or 2s because these tend to tell the buyer 'don't touch this with a barge pole'. The overall standard is not really that bad in the industry at the moment, so you will only see a couple of products come up as being this bad. I don't believe that anyone would really suggest that they really perceive there to be a massive difference between 1 and 2 star ratings, and therefore a simple 1 for anything below par would surfice.

That said, I can't see how you'd change it without asking all the reviewers to go back over their earlier reviews and re-score them. For many, particularly the staff reviewers who've done a hat full of reviews, this would be too big a job to be practical.

Just some thoughts...

Ben

 

[tensen]

Yes.. basically with a 5 point system, the majority of products should be between 2 and 4. With 1 being almost no redeemable component to the product. And 5 being an exceptional product. Since by definition everything can't be exceptional.... the majority of really good products should be getting 4s, if the scheme of things is correct. With a 2 going to things with a lot of glaring typos and game balance problems, as well as really horrible flavor.

 

[Scarogoth]

Plus, of course...

Some people are like me, and were taught that if you haven't got anything nice to say, then it's better to say nothing. The staff reviewers have a much more normal curve shaped slew of reviews.

 

[Arnwyn]

I agree with malladin. IMO, the scoring system as it is set up is very poor, and (as malladin says) too restrictive.

Another problem is that everyone has different ideas of what the different scores mean. Some people think that a "1" means it should be immediately burned, but others might think that a "1" is a very poor product, but that *any* d20 product still has something salvageable. Same idea for a "3" - some think that it means "not so great", while others think that it means a normal, average product. Some publishers have even been noted as complaining about getting a score of 3!

As to those who get flamed mercilessly for giving a product a lower review score? I'm mystified by that as well. IMO, I think that there is a somewhat vocal minority out there that gets upset if, heaven forbid, you are negative about a d20 product...

 

[Olgar Shiverstone]

The "star" rating really doesn't tell you much.

Most d20 Magazine Rack reviews translate over as 4-star reviews here at EN World, just because the rating system at d20 MR is drastically different.

If you want a real feel for the products, you have to actually read the reviews. At least with d20 MR, the text provides the majority of positive and negative feedback that the grading system doesn't.

 

[Crothian]

Most of my reviews are average or better becasue I buy products I like and actually do a little research into them to make sure I like them before I shell out the cash.

Another reason is there is more good products out there then bad ones. The d20 market has many good companies and writers turning out great product.

And finally, we don't seem to get many peopole who want to write a negative review. More then not someone will post to the boards why such and such book is no good, but they rarely write a review on it.

 

[JoeGKushner]

I guess my real question is what's the point of the thread?

"I was just curious. I've looking over some of the posted reviews and am surprised by the high ratings seemingly given to every product. The few times someone gave a low rating to a product, he was flamed unmercifully. Why bother with the lower ratings if nothing is ever going to drop below a 3? It just seems odd that every product reviewed should be average or above. Am I missing something? "

Your statement above doesn't say you disagree with the high ratings, for instance. You do mention people getting flamed, but my negative Troll Lord and Fast Forward comments haven't gotten me too flamed yet.

To me, the RPG industry is probably one in which it takes a certain amount of skill to thrive. Art, writing, editing, layout, format, originality all have their place in the grading scale.

To me, an average product has okay art (a subjective idea), okay writing (once again, fairly subjective), fair price (subjective), and brings something to the table. (subjective. For example, some love lots of fiction while I prefer little.)

Good product is above in some ways but doesn't take full advantage of the medium.

An excellent product is one I can start using right away with little difficulty. This however, may not fit everyone. For example, I loved the Treant book. It's bigger than previous Complete Guides. Has classes, templates, campaign hooks, levels for treants as player characters, NPCs, monsters and other goodies.

If you don't like treants though, what you rate the product? 3? 1?

Reviews are subjective by their nature.

This type of question comes up every couple weeks.

It is hard for people to believe that there most products are average or above?

Those people who find the ratings hard to swallow should step up and start doing their own reasoned reviews. None of this "This product has too many 4 stars so I'm rating it 2 stars crap." Have some information about what's in the product, hit the major points not only of the product, but of an RPG book in general, and tell the good and bad things. If after all that it's below average, the 3 star rating, so be it!

 

[trancejeremy]

I think Crothian pretty much nailed it.

And I would point out, those companies that consistently put out bad products, do get rated poorly. As Joe mentioned, Fast Forward is a good example. Now, if you listen to the Fast Forward people, people here have some sort of vast conspiracy against them. But the reality is their early stuff was really awful, and still, even now, some of their authors apparently refuse to use the 3rd edition D&D rules (fully, anyway. They still seem to use some 1st edition things from habit...)

Most companies are in the 3 to 3.99 range, which is average. The only company with a lot of products in 4 range, is I think Green Ronin, and the vast majority of their products are top notch.