What constitutes a good vs bad review?

[Jonny Nexus]

I hate the numerical rating aspect of writing reviews, which is why I prefer writing them for pyramid where it is just a straight discussion and analysis and analysis of products review without a rating scale.

I think that might be better for cases like this, allowing a reviewer to say something along the lines of: "People who like this kind of thing will like it a lot, but personally I think it stinks" and leave it at that.

 

[Luddite]

Going back to the "positve and negitive" review. Let me sort of demonstrate my "buying process."

-> Did Monte Cook write it? Yes, then it is on the "Will buy" list
-> Did MC publish it but not author? Yes, then on the "Most likely will buy"
-> Is this some cool Licensed Setting that I am a fan of? (ie B5) move to "Fan Budget"
Now that the auto-buys are out of the way...

It now depends on the topic of the book. In general, a product that scores a lot of 5's from "trusted" reviewers will get a serious look by me. And i would consider buying it even if I had no use for it.

When I am looking for some type of new rule system (like Naval Comabat) I will shop around. See which style I like. I will use the reviews to get a feel for the product, but I may choose the 3 over the 4 simply becuase I liked one particular aspect of that book.

I think I start ignoring products that start getting into the 2 range. Unless it is one of my Auto-Buys. (but then rarely do my Auto-buys get 2's )

So a 3 to me means. A solid product, but not something that I must have. Unless I am looking for information that it will be providing.

-gustavef

 

[RSKennan]

(To be more explicit:
1: This product is useless even if you are interested in the subject matter. To date, I have only ever awarded one "1" at ENWorld.
2: The product has some salvagable material, but has major weaknesses.
3: The product has strengths and weaknesses. Might be worth picking up if you are greatly interested in the subject matter.
4: The product is strong and makes me want to use it in a game. You should pick this product up if you are interested in the subject matter.
5: The product is imaginative and well executed. I would actively strive for ways to use this product in my game.)

This should be in a FAQ or on the reviews pages... it would clear things up for a lot of people. Honestly, though, this has been my immpression of the rating system all along. I do, however, feel that there is a common desire to buy only the best product of a given type, especially for people who operate on strict budgets, which may explain why some creators or publishers are unhappy with a 3/5. Not that that makes it right.

 

[Psion]

This should be in a FAQ or on the reviews pages... it would clear things up for a lot of people.

I should note that is my pesonal viewpoints. But considering some of the questions and incidents of late, I have already started putting together a "Psion's reviews FAQ" which I plan to put up.

 

[drnuncheon]

I think many reviewers who give out only 4s and 5s are entirely too generous. A 5 should be reserved for something really special, and a product that has some significant drawbacks along with some strengths should have a 3 not a 4, IMO.

To be fair, there are a lot of reviewers who give 4s and 5s because there are a lot of people who review stuff they buy, and they buy stuff they like. We don't get the free stuff.

J

 

[Utrecht]

Of course drnuncheon - that brings up the aspect of reviewers tilt - which most of the ENWorld reviewers attempt to address - but certainly could lead to skewing results. Further coupling this is the fact that the bad products tend not to get reviewed at all.....

Another question I have for the board at large is why do people hold a RPG product that gets a 2/5 in disdain - but then will happily go and see a movie rated 2/5? Why the dychotomy?

 

[drnuncheon]

Another question I have for the board at large is why do people hold a RPG product that gets a 2/5 in disdain - but then will happily go and see a movie rated 2/5? Why the dychotomy?

Well, that's pretty obvious - to me anyway.

The 2/5 rating on a game book represents the opinion of an amateur fellow gamer, who have at least that much in common with you, and more than likely have a lot more than that (see any of the 'gamer stereotype' threads for more details)

The 2/5 rating on a movie represents the opinion of a professional movie critic, who may have nothing at all in common with you beyond species.

J

 

[Graf]

It was not on this site It was on a gaming site not a d20 site. I have no problem with them telling you what you need to do to convert it but do not cut it down because of it.

Hope that clarifies that particular review.

Fair enough. If they weren't reviewing it as D&D consumer but simply as a product in the UA line thing they're being a bit silly.

 

[Merova]

Utility

Fair enough. If they weren't reviewing it as D&D consumer but simply as a product in the UA line thing they're being a bit silly.

Hi all!

As a reviewer, utility may be a factor in determining overall product value. IIRC, the review in question is over at the Gaming Report, written by Wayne Tonjes. It's a solid review that gives the product high ratings overall. However, there is an obvious caveat to the consumers:

The product is written for UA, and fully incorporates the system in even the smallest NPC and creature statistics. Therefore, its utility for the larger fantasy roleplaying audience is limited by the need to convert every mechanical modeling. If you're looking for a good fantasy scenario for your game, this product is only of high utility if you're playing UA.

This is a viable comment in a "descriptive" review, which acts as a "consumer report." In a "critical" review, this is a useless comment, because the critique doesn't relate to description, but to the ability of a product to meet its design goals.

"Descriptive" vs. "critical"; there are two types of reviews, with different functions.

Thanks for reading.

---Merova

 

[Arnwyn]

I should note that is my pesonal viewpoints. But considering some of the questions and incidents of late, I have already started putting together a "Psion's reviews FAQ" which I plan to put up.

Which very much surprises me. The numbers are there for all reviews on a standardized numerical scale, which (IMO) invites "quality comparisons" across products. However, the lack of a good description for what the numbers mean is pretty bad (the FAQ is quite weak in that regard, and the number of comments that pop up in reviews, especially in lower graded reviews, shows that). It's pretty clear that people interpret 1's, 2's, and 3's quite differently... and it becomes especially troublesome when some, um, "overzealous" ()review moderators wipe out those (admittedly rare) reviews that were confused over what the number ratings mean.

I think that "Psion's reviews FAQ" will be very useful in adding consistency (at least for as long as ENWorld insists on using the 1-5 numerical rating scale).