Are reviewers evil?

[baseballfury]

I think it's funny that Jim Ward seems to believe that EN World as a community is biased against his company. He makes it sound like we are some kind of alien groupmind and that we all agree about everything. Spend five minutes on these boards and you'll figure out that isn't true. People debate a huge variety of topics heatedly (but generally politely). It seems like he wants to blame us for the crappy reputation of FFE's products. From my vantage, if a community as large and diverse as EN World can come to a consensus that FFE's products are substandard (and I'm being kind with that phrasing), then FFE should take a long and serious look at its books. The simple truth is that there are a lot of companies that are better at making d20 books, from big players like Green Ronin and Fantasy Flight to smaller companies like Privateer and Fiery Dragon. If FFE wants to be taken seriously, they need to lose the arrogance and make some serious strides in quality control.

 

[Darrin Drader]

I think it's funny that Jim Ward seems to believe that EN World as a community is biased against his company. He makes it sound like we are some kind of alien groupmind and that we all agree about everything. Spend five minutes on these boards and you'll figure out that isn't true.

What? That's crazy talk! Haven't you ever heard of Hivemind? We're all in Pkitty's brain.

Or, as one of the great troubadores of all time said:

I am he as you are he as you are me and we are all together
See how they run like pigs from a gun see how they fly
I'm crying

Sitting on a cornflake waiting for the van to come
Corporation teeshirt, stupid bloody Tuesday
Man you been a naughty boy
You let your face grow long
I am the eggman, they are the eggmen
I am the walrus, goo goo goo joob

 

[Gez]

YES

Yes, reviewers are evil: They get free copies of books , while I don't . Therefore, they are evil.

 

[Dr_Rictus]

One thing he talks about that I agree with is reviews by people who haven't even played the game.

And yet, you really don't have to play in order to catch rules errors. And you can make a really informed evaluation about balance issues without playing, you just can't necessarily be right on each call.

Basically, you don't have to actually eat bad food to know that it smells. This is how FFE's products are actually evaluated on store shelves. Lord help them, for whatever reason, many of them as far as I can see are just ... not very good.

 

[Tiefling]

IMO it's fine to review a product without playtesting it, but you should at least tell the reader that you did so.

 

[Psion]

On the issue of "not reviewing products you have not played", I have a post on the FFE board about this point that sums up my feelings that I should repost:

(With respect to my position that you don't have to play every supplement to get a good review of it.)

The argument I sense coming is "well, then how can you know how it works?" In some cases this is a good point, and I have revised ratings upon playing a product. But this mostly applies to adventures. (For example, Nercromancer Games' Rappan Athuk played better than the quality of the book suggested it would, but their Demons & Devils adventure was pure drudgery.) Supplement style products tend to show their quality more on the surface, I think.

Some publishers see playtesting as mandatory for a review (this is the argument I get from Matt Colleville of Eden.) OTOH, many publishers ask reviewers to review products they receive within 30 days (because that is when they sell the most of a new product; RPG reorders are very soft these days.) Those goals directly conflict. I try to get reviews out the door ASAP, so this means I seldom get a chance to playtest an item before writing a review.

This being the case, many of my reviews only really reflect qualities of a product that are visible on the surface. I maintain that in most cases, this is not insubstantial. If I see, for example, missing information I would have to add before playing it, then that is every bit as significant for the purposes of play as it is for a review. On the other hand, good ideas (such as Dungeon World's strange entry and exit requirements and some of the cool tidbits in TQ:C&T) are apparent from a reading and a valuable contribution to a game.

The truth is that playtest reviews are better than "dry" reviews for revealing some aspects of a product. But if you rely only on playtest reviews, you will not have very many quality reviews and not have very many reviews soon after release. There simply isn't enough time to playtest a large body of products in time for the review to be useful to purchasers and publishers.

 

[WizarDru]

Playtesting also presents another challenge to reviewers. Even the most cursory of glances at the messageboards here will reveal that different gamers prefer different styles. Joshua Dyal, for example, prefers a radically different game than I do. Even if we use no house-rules and purely core material, we still have differences between games. Is magic plentiful, and are items purchasable in shops? Does your DM favor classed-NPCs over monsters? The list goes on and on.

This means that each review would have to have a qualifier section, or some sort of disclaimer. That detracts from the review at hand, IMHO. Worse, with the exception of adventures, playtesting becomes increasingly more problematic when the scale of a supplement increases or overlaps additional products. Many d20 supplements, however, are incorrect on the very face of the rules themselves, and no playtesting is necessary to realize this.

As for reviews of art...they are difficult, at best. When I was 12, I thought Erol Otus was terrible...now I consider him wonderful. Art is subjective...but poorly constructed art can be distracting. Otus, for example, had excellent composition and a unique style...the half-naked elf-chick in the quintessential rogue, however, isn't just embarassing...it's poorly drawn, to boot.

 

[Zappo]

To all people noting that you don't have to say something good in a review.

From reading Ward's article, I got the impression that what he meant was that you should say something good if you give a low or average score. Or, why saying only bad things and then giving 2/5? Either you say at least something good, or you give 0/5. I can see some sense in that. Reviews that do otherwise give me the feeling that the reviewer is working off his personal feelings instead of rationally, or that he doesn't use the rating scale properly.

I haven't bought or read anything from FFE, and I hardly read reviews of RPGs, and from my less-or-more external point of view I say that I agree with most of what Ward says. Whether he wrote it out of bitterness or not, whether he meant to do ad-personam attacks or not, I don't care, he has good points on how reviews should be made.

 

[Psion]

From reading Ward's article, I got the impression that what he meant was that you should say something good if you give a low or average score. Or, why saying only bad things and then giving 2/5?

Which is another instance of unsubstantiated allegations and false implications by Jim. As I am about the only person who gives FFE products an average rating (Jeff Ibach selectively fawns, just about everyone else pans them), I really have to feel like he is specifically talking about me here. That being the case, I challenge you to look at some reviews of products I have given average ratings and tell me that I did not highlight positive points:

Swords of Evil
"I do think that Swords of Evil is of much better utility than the hardbound "Items of Power" books. There is a much better ratio of background/flavor text to usuable text, and the authors lean more strongly on using the actual d20 System conventions instead of presenting the concepts much more generally as they did in the older books."

Enchanted Locations: Crypts & Tombs
"the ideas in here would make great adventure-starters for GMs who don't want full-blown adventures, but could use a few ideas."

Encyclopedia of Weaponry
"The Encyclopedia of Weaponry does present a number of interesting and unique weapons for use in the campaign, and the d20 statistics are generally solid, as are the additional rules and feats."

Wondrous Items of Power
"A few of the ideas are pretty good and do provide for the possibility of interesting and flavorful campaigning"

Enchanted Locations
"The maps would make decent adventure-starters, providing a few nuggets of ideas and giving your some elbow room."

Dungeon World
" found Dungeon World to be a rather refreshing idea, one that reminded me of various books and TV shows of my youth in which the main characters are heroes caught in a strange land. Though you could use it as a simple dungeon, the strange twists of the setting could make it so much more."

So, in short, while you may agree that you should not exclusively pan someone in an average review, I think Jim is being a bit disingenuous in suggesting that this is actually happening. If he weren't so blinded by his pride and busy bristling over the criticism, he might not have missed the positive comments in the review.

 

[JacktheRabbit]

I think it's funny that Jim Ward seems to believe that EN World as a community is biased against his company. He makes it sound like we are some kind of alien groupmind and that we all agree about everything.

To a small degree he is right. Too bad for him.

Jim Ward sounds very angry. He is very angry because he has been involved in DnD for decades and his name alone should be selling his products like hot cakes.

Unfortunately for Mr. Ward the OGL came out a decade too late. Imagine if the OGL had come out in 1990 or say the late 80's before the internet was what it is today.

Back then there was no ENWorld and most people had very little to go on when purchasing a product. You might here a bit from your fellow gamers at the store if you were social but beyond that you were on your own.

In a world like that Ward would sell well. He has a well known name and the facts about how poorly written his products are would get out very slowly as it would only be person to person word of mouth.

Everyone says that the Internet dumbs down this or that because it lets anyone comment at any time about anything. The internet also though allows the rapid dissemination of information. That balances things by us share information.

Bottom line? Jim Ward cannot sell on name alone because we talk.

I don't think he likes that.