A rant about D20 Magazine Rack

[Khur]

U Can Ree-VU 2

I think everyone is taking this way too seriously. (Did I just say that?) Let's lighten up.

Yep and a very poor example of how to review a product he uses one product as an example of what the other product should have been.

Very poor review standards if you ask me. Each and every review MUST be independent of other products.

Anything else is just beign a fanboy.

I'm not defending the specific review with my comments, but do you mean to say anyone comparing one product on the same subject to another product on the same subject is a fanboy? Strange. I, for one, want to know which product on a specific subject is the better for my limited gaming budget. In a non-gaming example, isn't it pertinent to know which sports car is better if one can only afford one or the other? If one wants an instruction manual on how to build a radio, isn't it important to know which manual best suits the project? If one wants to build a custom dwarf and toss him into play, isn't it good to know which book serves better? Of course, one has to be careful about comparing apples to apples, or noting significant differences between items (like price, page count, and so on). It is valid to compare similar products, and valuable.

The major problem with comparing Hammer & Helm to Heroes of High Favor: Dwarves is that one book is so much bigger and more expensive than the other. It could be argued that you just can't make a Hammer & Helm out of such a small book as HoHF: Dwarves.

Dwarves do not exist. They are a fictional creation in DnD pretty much copied straight out of Tolkien. A dwarf is a dwarf because he is short, carries an axe, and has a bad attitude.

Anything else is just a short humanoid. If you love dwarves but want something new then run a halfling or a gnome. A dwarf is a dwarf because of those old "sterortypes".

Just for fun here, but dwarves do exist--they are persons suffering from a genetic disorder called dwarfism. They have unusually short torsos and limbs and are distinct from midgets in that they are usually atypically proportioned. Midgets are also wee folk, but tend to be proportional for their size.

As for the fictional creation, D&D probably did lift them from Tolkien, but Tolkien lifted them from Scandinavian myth, a subject on which he was quite extensively studied. I'd also like to note that some Tolkien dwarves neither carried axes nor had bad attitudes. Scandinavian dwarves, on the other hand, tend to get stoned by sunlight.

Your second statement is a bit strange. Do you mean to say there's no room for innovation even in a stereotype? I say those who want to really play the stereotype should play Discworld dwarves! Now there's a living stereotype (and that Terry Pratchett is a hoot!).

All seriousness aside, though, a dwarf is not a dwarf because of stereotypes. That's like saying all Americans are the same. There's plenty of room for innovation, cultural exploration, and hairy, muscular lumps of crunchy goodness in the dwarf race, as Jesse Decker and Benjamin Durbin proved in both their works. Are all the dwarves played in your game exactly the same? If so, it sounds extremely boring.

Perfected dwarven roleplaying:

Belonging to the online comic PVP Online, of which I am a fan, but not the artist! Therefore it's (c) 2003 Scott R. Kurtz!

Anyway, I see no reason for the reviews to change. A look at many of them shows they're in line with the general opinion on this site. If that's true, what's the problem?

 

[JacktheRabbit]

Well I must say this came out of the blue!


And to open a thread, on another site, flaming a completely different site, without even mentioning it to the site you are flaming, asking questions, or e-mailing somebody? That is just, well, ...rude to put it nicely.

Rude would be jumping in on a topic you do not know anything about and saying something as stupid as this.

The reviews are here on ENWorld. The scores granted are on ENWorld.

Why would I go open a topic at D20 MR about a complaint I have about something they are doing at EN World? That is just dumb. I make the complaint HERE because HERE is where it needs to be fixed. I do not frequent D20 MR's site so I really do not care what they do over there. I do frequent this site on a regular basis and it is content here on this site that made me start this discussion.

 

[diaglo]

...insicurities pop-up...pop up...seam...seem...apperantly...peoples...gaurantee

please tell me you review your reviews before you post them.

If you don't like the reviews, don't read them.

how do i do this?

 

[Ghostwind]

Alright, I've had some sleep and time to think without my emotions interfering (stupid things - they tend to get in the way). The biggest problem I see is the perception of exactly what the numeric score represents. To me, it's a matter of straight scale. 5 is an A, 4 is a B, 3 is a C, 2 is a D, and 1 equates an F. Reading some of Psion's reviews (which likely is the standard everyone is judging us against), that seems to back up this line of thinking (Feats C+ =3, Sanctuary C+ =3, Beyond the Walls B- =4, Hallowed Might B =4, Necromancer's Legacy A- =5, and so forth). Previously, I had been trying to combine both the final d20 MR numeric score and the letter grade our reviewer assigns. Sometimes it is an easy match (i.e. both equal a B), other times it is not (score indicates a C, but reviewer gives it a B).

What I will do is strictly use the letter grade the reviewer assigns since that represents the common ground of both systems. What does that mean for scores here on EN World? It means you will continue to see 4's and 5's, most likely. Face it people, the level of quality is definitely better than it was even one year ago. Books are going to rate higher because the product is better.

So now you have to ask yourselves, is the issue here really the so-called "bias" that my staff exhibits by awarding higher scores or is it because the thought of seeing scores that award quality is too much because it somehow makes the review database less legitimate? It is entirely possible to give a book a good overall score and yet be critical of one or two points. The individuality of every person (including reviewers) means that the interpretation and assessment of the products are going to vary. Just because I like Airships doesn't mean you will. Likewise, just because you like HOHF: Dwarves doesn't mean I will or do.

As I said before, maybe those dissatisfied with what they are seeing in terms of the scores should make a concerted effort to get more people involved in review writing. If 25 people write a review of Feats by AEG and all but one find it good enough to rate a 4, then there is a undeniable concensus that it is a good book. However, if one person rates it a 4 while everyone else gives it a 3 or lower, then that one person sees something in the book that stands out for him that others do not. The majority would indicate that the book is fair in quality, but that one person sees it as a good product.

The review is an opinion of that product as written by the reviewer himself. Some reviews are more critical than others and likewise, some reviewers are more critical than others. This is universal regardless of where the review may be posted or read. This is why it is so important to read the review itself rather than making a snap judgment based upon the score of the product. I ask everyone who has openly criticized myself and my staff about the reviews here, have you actually read all 185 reviews or are you simply looking at the score and judging us because of that? I will be the first to say that not every review is Pulitzer material, especially some of the older ones from a year or so ago. But I'll also say that my current staff represents some of the best reviewers around.

Going back to Doc's very first post, I wonder if the issue is the number of reviews that get posted at once?

D20 Magazine Rack has flooded the reviews again. What a surprise 21 reviews by D20 MR and what is the range? 19 of them got 4's and two of them got 3's.

This got me curious and I did some quick checking.

D20 MR has done 185 reviews so far.

117 of them have been 4's.

33 of them have been 3's

32 of them have been 5's

3 of them were 2's

What does this tell me? Absolutely nothing. The reviews give equal ratings to products that I have found (and my friends have found) to be nowhere close to the rating that D20 MR has given out.

If you go into the database and look at the Full Publisher List that breaks things down by product and then lists the reviewers and the scores of that particular product, for the most part, our reviews are in line with everyone else's.

For example:
*Undead by AEG (Simon 3, d20MR 3, Psion 4)
*Toolbox by AEG (Psion 4, d20MR 4)
*Oathbound by Bastion Press (Simon 3, d20MR 4, Gamewyrd 5)
*Path of Magic by FFG (d20MR 5, Simon 4, Sigil 4, Crothian 4, shadeus 5)
*Wrath & Rage by Green Ronin (Illuminati 5, d20MR 4, Simon 3, Madfox 4, trancejeremy 4, Psion 4, Gamewyrd 4, JoeGKushner 3)
*Fury in the Wastelands by Kenzer (d20MR 4, Psion 4, Simon 4, JoeGKushner 4, outlaw 5)
*Broadsides! by L.I. (d20MR 4, Psion 4, Simon 4, voadam 4)
*Beyond the Walls by Mystic Eye (d20MR 4, Psion 4, Simon 3, Gamewyrd 3)
*Book of Vile Darkness by Wotc (Gamewyrd 3, d20MR 4, Sigil 1, Henry@home 4, Isida 4, Crothian 4)

Now I ask, how are we unfairly biased in our scoring? It looks like we are fairly close to others opinions on most products. Look at the big picture of all reviews of that product before saying we are nothing more than the "mouthpieces of the industry".

Finally, if you disagree with a review because you don't believe it represents the score YOU think it should have, is it fair to ostracize the reviewer into not writing anymore reviews? It happens quite a bit around here from what I have seen. New reviews are being posted by people who are at least making the attempt and they are often targeted for deletion because they come across as either "fanboy" or too short. Why not create a quick little "form comment letter" that can be stuck in the comments pointing out some of the essential points a review should include such as an assessment of both good and bad points? You could also set a minimum word count both in the letter and on the d20reviews main page. Both would give a sense of trying to be helpful rather than "this isn't a review and we should delete it immediately." Review writing isn't easy and not everyone can do it successfully on the first few attempts. Work with them rather than smacking them down.

 

[Psion]

To me, it's a matter of straight scale. 5 is an A, 4 is a B, 3 is a C, 2 is a D, and 1 equates an F. Reading some of Psion's reviews (which likely is the standard everyone is judging us against), that seems to back up this line of thinking (Feats C+ =3, Sanctuary C+ =3, Beyond the Walls B- =4, Hallowed Might B =4, Necromancer's Legacy A- =5, and so forth).

Heh... never would have guessed that my little rating comment come under such scrutiny.

At any rate, yes, that is how I see the 1-5 scale. Some people seem to be under the impression that the statistical average of scores on ENWorld should be 3, and that is just not practical. Only the harshest of professors in college that I knew would inflict a quota on grades handed out.

By showing the equivalence to the standard grading system, I hope to relate to a scale people are more familiar with; some people think that a 3 is a "bad" or "failing" review; I see it as a passing grade. Some people howl angrily if you give a product that they like a 3. I resist such characterization because I need room to be discerning.

However, that is sort of a secondary effect. The primary reason that I threw some + and - grades scores in is that a lot of times, I waffle about what to give a product, and I want the audience to be able to see that.

 

[Morrus]

Rude would be jumping in on a topic you do not know anything about and saying something as stupid as this.

Did you read what I said earlier? If not, go back and read it now.

Somehow I knew it would be you who couldn't resist having the last word. Keep it nice, please.

 

[JacktheRabbit]

Did you read what I said earlier? If not, go back and read it now.

Somehow I knew it would be you who couldn't resist having the last word. Keep it nice, please.

Sorry, but after about the 5th time I get told that I am being rude or trolling for having this discussion here (where the damn reviews I have a problem with are located to begin with) I start to lose patience.

I don't give a rats behind what D20 MR does on its own site. My problem is their porting over reviews and not bothering to create a proper rating for the product.

 

[Khur]

Sweet!

Alright, I've had some sleep and time to think without my emotions....

Ah, the sweet voice of sanity and reasoned analysis. Ambrosia! This one post did more to diffuse the problem here than any other three posts, especially that bit about the grading being on par with everyone else's. Excellent!

Well, now that that's cleared up....

 

[Ghostwind]

Heh... never would have guessed that my little rating comment come under such scrutiny.

At any rate, yes, that is how I see the 1-5 scale. Some people seem to be under the impression that the statistical average of scores on ENWorld should be 3, and that is just not practical.

We are in agreement on this then. For example, the numeric score of a 4 may represent a letter score ranging from a B+ to a B-, thus providing the leeway to be flexible.

By showing the equivalence to the standard grading system, I hope to relate to a scale people are more familiar with; some people think that a 3 is a "bad" or "failing" review; I see it as a passing grade. Some people howl angrily if you give a product that they like a 3. I resist such characterization because I need room to be discerning.

Perhaps the inherent problem lies with the words attached to each score. The score of 3 indicating "average" creates a more unfavorable image than the word "good", yet both can mean the same under the right circumstances. Maybe the words associated with each score could be modified to something along the lines of "5-superb, 4-excellent, 3-satisfactory, 2-problematic, 1-appalling"? Or perhaps the words could be replaced with the simple letter grade associated with that score: "5-A, 4-B, 3-C, 2-D, 1-F"?

Then has been more than one discussion about how the current EN World review scoring system has its problems and perhaps something as simple as a slight change in labeling may shift the whole perceptive balance about what the scores represent.

 

[TempesTMR]

please tell me you review your reviews before you post them.

I have written like maybe 3 reviews . .and that's what editors are for! Unless you meant the post, then no, I did not review it very well obviously!

At three in the morning I'm about whooped, and could have cared less if I spell the same word 12 different ways. lol

how do i do this?

Hmmm, I forget. I'll get back with you though! Promise . .