• NOW LIVE! Into the Woods--new character species, eerie monsters, and haunting villains to populate the woodlands of your D&D games.

A rant about D20 Magazine Rack

Status
Not open for further replies.
blackshirt5 said:
I somewhat agree. While I think that it's appropriate to compare and contrast products(for example, if I were to review the Quintessential Paladin once I get it, I'd surely draw comparisons between QP and Defenders of the Faith, because they're both Paladin splats). What I don't think is appropriate is saying things like "This is what Heroes of High Favor: Dwarves should have been" is appropriate in the Hammer and Helm review.

Also, I still contend that lowering something's score(such as the 3.5 both books got because they're "primarily restricted to dwarves") when reviewing a race or class splatbook is wrong.
Agree on both points.

IMO, products can and SHOULD be compared to other products, with the following factors weighted in (in approximately this order):

1.) How much they overlap on subject matter.

2.) How close their release dates were to one another. Later products should have "learned" from mistakes in prior products - including those put out by other publishers.

3.) Price/size

For example, Hammer & Helm, the Quintessential Dwarf, and HoHF: Dwarves all came out at almost exactly the same time. That means I have every right to compare them, bearing in mind that HoHF is shorter and cheaper than the other two, and so should be able to have less total "stuff" in it without being penalized. H&H and QD, being nearly identical in size and price, should easily be comparable. That doesn't mean I should look for something in H&H and try to find the same thing in QD to see whose is "better." It means I should try to compare the total breadth of options offered, in diversity and utility, and compare that breadth - and if there ARE areas where they happen to try to do the same thing, only then should I try to figure out which is "better" - but keep that within the confines of "assuming I use all the other rules in the book it came from only." IOW, I can't say, "mix and match X, Y, and Z from QD with A, B, and C from H&H." It does mean I can say, "X in QD was done as B in H&H and I liked X (or B) better."

However, I should take more care when comparing, say, Path of Faith to Defenders of the Faith. The price is somewhat comparable, but the fact that Path of Faith had an extra year or so to "sit back and watch" the evolution of the d20 world means I should hold the material therein to a higher standard than DoF. PoF should have learned from DoF's mistakes.

Similarly, I should not hold the Creature Collection (first edition, not revised edition) to the same standard as the Fiend Folio... because the FF has had three years to "learn from the mistakes" of the CC and other works.

I should not really compare Alchemy & Herbalists with Darwin's World as the subject matter is different.

Lowering a score based on lack of overall applicability should only be done when it's not fairly clear that the book is directed at one specific niche. While I shouldn't penalize the Quint. Paladin for not giving me options for evil characters, I might well penalize, say, "Bob's Super Guide to Prestige Classes Various and Sundry - Suitable for Everyone" for having almost no PrCs for non-magic-users and zero PrCs that a character of NG alignment could take (an extreme example, to be sure LOL).

--The Sigil
 
Last edited:

log in or register to remove this ad

Did you ever stop to think...

Has it occured to anyone that if all you're reading is the number, the problem isn't with us, it's with you? I do this out of my love for the hobby and the faint hope that someone, somewhere finds my thoughts helpful. Since all you read are the numbers, obviously, that hope is wasted here. You all seem to think it's easy to write a good review, but have you tried it?

A review is an opinion and I try to offer the best and least biased opinion that I can. I don't bash something because it's a Wizards product, nor do I offer praise to a product just because it's from Bastion Press (a company I've grown much fonder of in recent months). I try to present a good picture of the product while picking out the good and the bad points. Sometimes I do better than others. That's because some weeks, I just don't have as much time. See, I work a full-time job, support a family, and try to fit in one night a week for recreation.

You all talk real big, but when I don't see feedback, I tend to assume that everything is alright. If you disagree with me, present a logical argument to my review in the space provided. I'll respond and I'll support my reasoning or else I'll offer an apology if you spotted something I missed. You can make the reviews better by simply putting your two cent's worth in. But instead of doing that, you'd rather openly and publicly flame us.
 

Ghostwind, and all the other d20 MR reviewers - despite the guff you've been getting, I've found your reviews generally helpful (when I read them). There have been a few times when your reviews have been the only ones available for a recent product I was interested in, and in those cases, I always received enough information to make my purchasing decision.

And really, what more can you ask for from a review than that?

However ....


D20 MR has done 185 reviews so far.

117 of them have been 4's.

33 of them have been 3's

32 of them have been 5's

3 of them were 2's


After reading d20 MR's reviewing standards, I feel fairly safe in saying this is due to a compressed and inflated scoring scheme that actually centers around the 4 point mark. From d20 MR's standards page (which is full of very useful advice, but did NOT have a readily apparent link from the most recent Airships review) :


Scoring (as it applies to the d20 Magazine Rack)
Open Game Content: The scoring on this is a little subjective but use this as a guide...
*5.0 - The entire contents except for graphic elements are ogc.
*4.5-4.0 - Most of the contents except for an occasional section of single chapter are ogc.
*3.5-3.75 - Only the game mechanics themselves are ogc. Spell names, monster names, class names, etc. are closed.
*<3.0 - There is very little game mechanics present (less than 5% of the book).
*0.0 - No open content (Wizards of the Coast, Kenzer, Arthaus, and other licensed settings from Wotc).

d20 Compliance:
*5.0 - Everything follows proper game mechanics including skill points, feats, game balance and other actual rule functions.
*4.0 - There are a few minor errors (less than 10) or only one major error.
*3.0 - A significant number of minor errors or few major errors exist.
*<3.0 - The book has serious game mechanics and rules issues.

Originality:
-This is a judgment call. Most scores will fall into the 4.0-5.0 category unless it is something that has been done time and time again (such as a new version of a Star Trek game).

Playability:
*5.0 - Useable by everyone, player and GM.
*4.0 - Primarily either player or GM, but still somewhat useable by the other.
*3.0 - Either player only or GM only. Or devoted solely to one particular class or race.
*<3.0 - Very restrictive as to who can gain the most use from it.

Value for the dollar:
-Again, a judgment call.

*5.0 - Good value for all.
*4.0 - Reasonable investment but better if on sale.
*3.0 - Only purchase at full price if you truly want or need it, otherwise discount sales only.
*<3.0 - You have to really want this book, because it's overpriced for what it offers.

I think a better system might be:
<table>
<tr>
<th>Category\Rating</th>
<th>0.0</th>
<th>1.0</th>
<th>2.0</th>
<th>3.0</th>
<th>4.0</th>
<th>5.0</th>
</tr>
<tr>
<td>OGC</td>
<td>No open content (Wizards of the Coast, Kenzer, Arthaus, and other licensed settings from Wotc).</td>
<td>There is very little game mechanics present (less than 25% of the book).</td>
<td>Large portions of the book, especially the most innovative and/or useful parts, are NOT OGC.</td>
<td>Only the game mechanics themselves are ogc. Spell names, monster names, class names, etc. are closed. (This is currently the deFacto standard in the industry.)</td>
<td>Most of the contents except for an occasional section of single chapter are ogc.</td>
<td>The entire contents except for graphic elements are ogc.</td>
</tr>
<tr>
<td>d20 Compliance</td>
<td>Is not a d20 product.</td>
<td>Is for an ancestor of the d20 gaming system (1st or 2nd Edition ADnD).</td>
<td>The book has serious game mechanics and rules issues that actually interfere with game play.</td>
<td>The book has serious game mechanics and rules issues, but these do not interfere with gameplay, or are easily compensated for.</td>
<td>There are a few minor errors (less than 10) or only one major error.</td>
<td>Everything follows proper game mechanics including skill points, feats, game balance and other actual rule functions.</td>
</tr>
<tr>
<td>Originality</td>
<td>Not applicable</td>
<td>An unupdated reprint of a previously issued product.</td>
<td>An updated version of a previously issued product. E.g. A 3e revision of the Basic DnD module B1</td>
<td>A new product that uses all the standard tropes and cliches of the genre in an unimaginative manner</td>
<td>A new product that uses some of the standard tropes and cliches of the genre in an imaginative manner</td>
<td>A new product that uses few, if any, of the standard tropes and cliches of the genre.</td>
</tr>
<tr>
<td>Playability</td>
<td>Not applicable</td>
<td>The product is extremely setting-specific, or is otherwise very difficult to incorporate into a game. It requires extensive adapation before use.</td>
<td>The product is setting-specific, or is otherwise difficult to incorporate into a game. It may require extensive adapation before use.</td>
<td>The product can be incorporated into any game with a reasonable amount of preparation.</td>
<td>The product can be incorporated into any game with a minimum of preparation. Most "drop-in" adventures will fall here.</td>
<td>The product can be incorporated seamlessly into any game with no preparation at all.</td>
</tr>
<tr>
<td>Value for the Dollar</td>
<td>Not applicable</td>
<td>You have to really want this book, because it's overpriced for what it offers.</td>
<td>Only purchase at full price if you truly want or need it, otherwise discount sales only.</td>
<td>Reasonable investment but better if on sale.</td>
<td>Good value for all.</td>
<td>Well-worth the price.</td>
</tr>
</table>
 

Re: Re: A rant about D20 Magazine Rack

Attempting to make that visible.

GuardianLurker said:
I think a better system might be:<table><tr><td><font color="#FFE078">Category\Rating</font></td><td><font color="#FFE078">0.0</font></td><td><font color="#FFE078">1.0</font></td><td><font color="#FFE078">2.0</font></td><td><font color="#FFE078">3.0</font></td><td><font color="#FFE078">4.0</font></td><td><font color="#FFE078">5.0</font></td></tr><tr><td><font color="#FFE078">OGC</font></td><td><font color="#FFE078">No open content (Wizards of the Coast, Kenzer, Arthaus, and other licensed settings from Wotc).</font></td>
<td><font color="#FFE078">There is very little game mechanics present (less than 25% of the book).</font></td>
<td><font color="#FFE078">Large portions of the book, especially the most innovative and/or useful parts, are NOT OGC.</font></td>
<td><font color="#FFE078">Only the game mechanics themselves are ogc. Spell names, monster names, class names, etc. are closed. (This is currently the deFacto standard in the industry.)</font></td>
<td><font color="#FFE078">Most of the contents except for an occasional section of single chapter are ogc.</font></td>
<td><font color="#FFE078">The entire contents except for graphic elements are ogc.</font></td>
</tr>
<tr>
<td><font color="#FFE078">d20 Compliance</font></td>
<td><font color="#FFE078">Is not a d20 product.</font></td>
<td><font color="#FFE078">Is for an ancestor of the d20 gaming system (1st or 2nd Edition ADnD).</font></td>
<td><font color="#FFE078">The book has serious game mechanics and rules issues that actually interfere with game play.</font></td>
<td><font color="#FFE078">The book has serious game mechanics and rules issues, but these do not interfere with gameplay, or are easily compensated for.</font></td>
<td><font color="#FFE078">There are a few minor errors (less than 10) or only one major error.</font></td>
<td><font color="#FFE078">Everything follows proper game mechanics including skill points, feats, game balance and other actual rule functions.</font></td>
</tr>
<tr>
<td><font color="#FFE078">Originality</font></td>
<td><font color="#FFE078">Not applicable</font></td>
<td><font color="#FFE078">An unupdated reprint of a previously issued product.</font></td>
<td><font color="#FFE078">An updated version of a previously issued product. E.g. A 3e revision of the Basic DnD module B1</font></td>
<td><font color="#FFE078">A new product that uses all the standard tropes and cliches of the genre in an unimaginative manner</font></td>
<td><font color="#FFE078">A new product that uses some of the standard tropes and cliches of the genre in an imaginative manner</font></td>
<td><font color="#FFE078">A new product that uses few, if any, of the standard tropes and cliches of the genre.</font></td>
</tr>
<tr>
<td><font color="#FFE078">Playability</font></td>
<td><font color="#FFE078">Not applicable</font></td>
<td><font color="#FFE078">The product is extremely setting-specific, or is otherwise very difficult to incorporate into a game. It requires extensive adapation before use.</font></td>
<td><font color="#FFE078">The product is setting-specific, or is otherwise difficult to incorporate into a game. It may require extensive adapation before use.</font></td>
<td><font color="#FFE078">The product can be incorporated into any game with a reasonable amount of preparation.</font></td>
<td><font color="#FFE078">The product can be incorporated into any game with a minimum of preparation. Most "drop-in" adventures will fall here.</font></td>
<td><font color="#FFE078">The product can be incorporated seamlessly into any game with no preparation at all.</font></td>
</tr>
<tr>
<td><font color="#FFE078">Value for the Dollar</font></td>
<td><font color="#FFE078">Not applicable</font></td>
<td><font color="#FFE078">You have to really want this book, because it's overpriced for what it offers.</font></td>
<td><font color="#FFE078">Only purchase at full price if you truly want or need it, otherwise discount sales only.</font></td>
<td><font color="#FFE078">Reasonable investment but better if on sale.</font></td>
<td><font color="#FFE078">Good value for all.</font></td>
<td><font color="#FFE078">Well-worth the price.</font></td>
</tr>
</table>
 
Last edited:



Yeah, I see the problem. A D20MR score of 3 is a BAD score, equivalent to a 2 (or maybe a 1) at EN World. Their score range is realistically from 3-5, not from 1-5.

Which is absolutely fine. The problem is you can't just move that score over to another place which uses a different system. To take the example to the extreme (and, yes, I know it's silly), what if EN World used a % score? You wouldn't still give it a score of 4 then - that'd be daft.

So the very simple solution is this - when D20MR posts a review at EN World, it uses EN World's scoring system, not D20MR's. On their own site, the same review uses D20 MR's own scoring system.

Thus, a score of 3 translates to a 1 or 2; a score of 4 translates to a 3; a score of 5 translates to a 4 or 5. They don't translate directly across.

The system shown above is essentially a reworking of D20MR's system into a 1-5 scale instead of a 3-5 scale, which would translate across to EN World just fine. Well, nearly - that above is 0-5, so it's slightly different.
 
Last edited:

Standards

D20 Magazine Rack is one of the few zines that have adopted standards for their reviewers. Every review must meet these standards in order to be published. They were nice enough to share their standards with the manufacturers and I was impressed at the lengths that they went to in order to provide a fair and balanced review.

Regards,

Rob Stone
President
Citizen Games LLC
 

Well I must say this came out of the blue!

I seldom come to the ENworld forums. Not that they are bad, actually pretty fun, but when you get this many people, insicurities pop-up, and they are better just avoided. Although I can always find something i'm looking for, good job there!

But then things like this pop up. Kinda odd really. You guys really don't seam to notice how hard it can actually be for reviewers to do a good job. And you are not even paying attention to what the reviews say (which is everything) you look at the score. Which is totally pointless. People like numbers, which is why they are there, but they are not the review, what is typed out, and described is the review. If you don't have the time to read them, then I'm sorry, but your complaints here seem useless on my stand point.

Reviewers are in a no-win situation. If they give low scores, they are flamed. High scores and they are biased. Average scores, and people complain cause there are to many average scores. This makes no sense to me, but preach your own I guess.

Morrus said it best above. The d20MR scores do not translate the best. And apperantly to please you all, they should. I'm not sure why, it must be peoples fascination of numbers, but you should read the review, that is the heart of everything. From most of what I have seen, I think we have one of the best scoring techniques available. By just looking at the score of each section you can get a good idea of what the book is, compared to a single number. But with 5 different area's, it's almost a gaurantee most will end up with an average score, that's just the way it works out. Somebody mentioned the whole "bell curve" above.

So I will leave it with a few simple idea's. Most importantly read the reviews, not the numbers, that is obvious. If you don't like the reviews, don't read them. And if you think you can do better, then start writing reviews, but don't get mad if we call you biased for high scores, or wrong on low scores. Maybe even be upset if most of them are average.

And to open a thread, on another site, flaming a completely different site, without even mentioning it to the site you are flaming, asking questions, or e-mailing somebody? That is just, well, ...rude to put it nicely.
 

TempesTMR said:
Morrus said it best above. The d20MR scores do not translate the best. And apperantly to please you all, they should. I'm not sure why, it must be peoples fascination of numbers, but you should read the review, that is the heart of everything. From most of what I have seen, I think we have one of the best scoring techniques available. By just looking at the score of each section you can get a good idea of what the book is, compared to a single number. But with 5 different area's, it's almost a gaurantee most will end up with an average score, that's just the way it works out. Somebody mentioned the whole "bell curve" above.

So I will leave it with a few simple idea's. Most importantly read the reviews, not the numbers, that is obvious. If you don't like the reviews, don't read them. And if you think you can do better, then start writing reviews, but don't get mad if we call you biased for high scores, or wrong on low scores. Maybe even be upset if most of them are average.

Sorta. I agree that people shouldn't put too much weight on the scores. However, given that people do notice them (and yes, I'm sure they read the reviews too - this hyperbole about just looking at scores and not reading reviews is exaggeration in the extreme), and given that the "fix" is so easy and effortless, why not adopt it?

Keep the same scoring system at D20MR but when posting a review to EN World, use EN World's scoring system. That way it all makes sense wherever you see the review, and it's so easy to do, and keeps everyone happy :)

Anyway, this thread is starting to get hostile. I think it's run it's course. I'll leave it open in case someone has something new and constructive to add, but I'm keeping an eye on it.
 
Last edited:

Status
Not open for further replies.

Into the Woods

Remove ads

Top