Question for the reviewers (open to anyone else too)

[BiggusGeekus]

I read Alan Kohler's review on rpg.net for the Twin Crowns d20 game:

http://www.rpg.net/news+reviews/reviews/rev_5781.html

He notes that the game is an excellent value on a dollar per page basis. While I have no doubt the game deserved the 2/4 rating the product recieved; it did raise some questions.

Clearly they could have added better art than the "lackluster" stuff Mr. Kohler saw ... but at an increase to the price. Should they have done that? When reviewing is price taken into consideration? Should price be taken into consideration? After all, there is some stuff out there on the net that is free, but I wouldn't waste my valuable download time on it when I could be reading d20review.com

I realize that this is a very open-ended and vague question. I realize there is no "right" answer. I'm just curious about opinions.

 

[enrious]

I personally think price does matter because I think the reviewer is trying to convey the *value* of the product, whether it's free or $40.

If there's a free downloadable product that is worthless then that's less of a problem than a $25 printed product that is way overpriced or not a good deal at any price.

Like I said, I'm interested in the value of a product; cost figures into the equation.

As an example, I looked at the price tag on the FRCS and thought $40 was outrageous. Now I think I would maybe have spent $10 more and felt satisfied.

On the other hand, there have been some $15-20 products that I would have felt cheated if they had been $5 .pdfs.

 

[Psion]

I think price matters... but only so much. Which is to say I weight it, but not heavily. The two highest price per page value products I know of (Twin Crowns and Mystic Warriors) only got 3's from me, simply because it is only one factor. If a book is good in some of the other factors I use in weighing books, the 2 or 3 or 10 more dollars it takes me to get it is worth it.

Just to give you an idea, sometimes when I don't know exactly how I want to rate a product (when I say to myself "is this a 3 or a 4?" or whatever), I fall back on a quick tally system that runs like this:

Rate each of these five categories +1, 0, or -1 (+1 strong, -1 if weak):

- Quantitative Value (vs. products in same range)
- Presentation
- Ideas / Inspiration
- Utility
- System Fidelity

Total it up. +2 is a 4. +4 or higher is a 5 (you will notice I give very few fives.)

Frex, for Twin Crowns I think I broke it down along these lines:

QV +1 (top notch)
P -1 (blase)
I 0 (didn't inspire me)
U +1 (lots of source materials)
SF 0 (Overall decent, but have big balance concerns about divine feats)

Total +1. That's a 3. Almost a four.

Now let's take the last product I purchased.

BoEM II
QV 0 (bout normal for a PDF)
P +1 (nice)
I +1 (soul magic was a great idea with loads of possible uses. The spells and items are also quite interesting)
U +1 (loads of new, usable skills, feats, and other mechanics)
SF +1 (It's monte...)

Hmmm... could be a 5 if I don't change my mind before the review.

Edit: +4 or higher is a 5.

 

[BiggusGeekus]

Psion, thank you very much for replying.

Your analyisis seems very fair and I'm glad I have a good idea where you are coming from. In all honesty it would seem that -- judging by your criteria -- a new publisher would be best off breaking into the market via PDFs or going all out and having a high production/cost book. However, as you note, the words matter. I also must admit that I like your "System Fidelity" aka "Heart" metric. I have long felt that there should be some room for a reviewer to openly "bump" a product. After all, B2: Keep on the Borderlands will always retain a soft spot in my gut, and that should be reflected in any commentary I give for that product.

Thanks again and happy gaming!

 

[Desdichado]

Total it up. +2 is a 4. +4 or higher is a 4 (you will notice I give very few fives.)

I, on the other hand, give many fours and fives. Then again, you review almost everything that is released: I only buy stuff that I'm fairly certain I'm going to like. It'd be odd for me to review something badly, because I'd know (hopefully) before I buy it that it isn't something useful to me.

 

[Psion]

I also must admit that I like your "System Fidelity" aka "Heart" metric.

Do you understand what I mean by "system fidelity"?

I pretty much mean "using the rules well." i.e., few or no mechanical glitches, doesn't make up new things when existing things can do the job, etc.

I have long felt that there should be some room for a reviewer to openly "bump" a product. After all, B2: Keep on the Borderlands will always retain a soft spot in my gut, and that should be reflected in any commentary I give for that product.

Hopefully that sort of thing will come through in the inspiration section. But I really don't think it deserves more than that.

 

[enrious]

Psion, I understood your system fidelity rating to be "adherence to the rules."

Thanks for the insight on your reviews and also to BiggusGeekus for the question.

It made me think.

 

[Psion]

Psion, I understood your system fidelity rating to be "adherence to the rules."

Well, it is that, but not just that.

I mean goofing up stat blocks, assigning arbitrary HP and HD types, out to lunch CRs, spells too powerful for the level, and classes or feats that give you in-game adventages for role-playing disadvantages or by penalizing thing you will not use vice drawing it from your resource pool (class abilities, feats, skills) are obvious things that will earn a product a "weak" from me in a category.

But it's not just adhering to rules, but using them well. For example, creating a subsystem that is new but consistent with other stuff that has been done, creature statistics and tactics that mesh well with feel that the product is trying to create, attention to appropriate placement of challenges and consciousness of their impact, and addressing a need in a novel or clever way are all factors that will tend to earn a product a strong in my system fidelity category.

Take, for example, Libram Equitus. One could have made a character that is good at archery and bad at melee by slapping on a bonus with missile weapons and penalty to melee weapons. Effective, but clumsy. Now what he did was give his archery classes medium bab advancement and gave the class a class ability that gave them a competance bonus with missile weapons that let them edge out a fighter. This was clever and worked within the system vice brute-forcing it.

Unfortunately, he lost some of those points by making multiple classes that used the same type of scheme that were obviously intended to work together, but never stating the bonus stacked.

 

[Psion]

Your analyisis seems very fair and I'm glad I have a good idea where you are coming from. In all honesty it would seem that -- judging by your criteria -- a new publisher would be best off breaking into the market via PDFs or going all out and having a high production/cost book.

Well, a publisher's economics are entirely an issue for the publisher. That's why I can't STAND when some company drone comes out and explain why their product is so expensive and names market considerations, etc. One such person tongue lashed me for knocking them down for putting out an expensive hardback with a very high price/page ratio with the excuse that his company cannot afford the larger print runs YADDA YADDA YADDA. The customer doesn't care. The customer doesn't receive value for your print run size. He just gets content X for value Y, and that's it.

That said, I try not to make one publishing category better. When I do my price analysis, I base it on comparisons to similar products based on the price or size. Relics & Rituals will get bounced against FFG black & white hardbacks. Gar'Udok's will get bounced against BoEM, and so forth. Otherwise, the general trend is that bigger books are cheaper per-page. That doesn't stop some of the smaller books from being worthwhile products.

There is one exception. After having seen only two books in the category of "small (<100 pages) color book for more than 20 dollars" (Monsters of Faerun and Minions), I think that the category in general is not cost effective.

 

[Numion]

I for one don't like reading reviews that give fives to products that are just 'OK and free'. It's not like the product was suddenly more useful to me, just because it's free. I have so much gaming material anyway, that something being free/cheap doesn't make the material any more likely to show up in my games. Only the quality matters.

And I also don't like any of the reviews that are just a couple of lines long. Despite the fact that the reviews are free themself.