Andy Collin's comments re censoring playtester reviews

TerraDave · Jan 31, 2008

...sigh...

Maggan · Jan 31, 2008

I think some of the reactions to the "please stick to the good things when on message boards" request from Andy proves a theory I have:

WotC should just say nothing at all.

That way, there will be nothing to rail about except the silence, and that railing would be over in say a month or two. Now we get new tidbits to rail against every so often.

"Don't fuel the fire" is my advice to WotC. Just keep quiet, and release the game and then start up the marketing for real.

/M

Dragonblade · Jan 31, 2008

zoroaster100 said:
I just read Andy Collins' comments quoted on ENWorld's front newspage today explaining what playtesters were and were not allowed to say. I must say I am disappointed with WOTC's decision to try to provide skewed information about the game by allowing certain playtesters to comment but only if they have positive comments. I still trust that the freelancers who spoke out do honestly like the game, just as they stated, but now we know if there were four times as many freelancers who had nothing positive, and much negative to say, those people were forbidden from speaking out. I believe that is deceptive and manipulative on the part of WOTC. I'm sure there are many other companies that do similar marketing practices, but that doesn't make it right.

I read Andy's defense for the practice, which is essentially that there is no point in having negative comments shared with consumers at this point because the criticism can just be used internally to fix the problem. But that doesn't strike me as a legitimate defense. From what we've heard, most of the rules at this point are pretty much done. So if someone felt overall the game is not as fun with the new rules, I don't see how that could be fixed at this point. More importantly, if the game is still so in flux that negative comments are not relevant because things might get fixed, the same could be said for positive comments.

I still have high hopes that the fourth edition rules will be fun. But the credibility of WOTC's marketing department has taken a beating for me this day.

Your position is completely unreasonable. NO COMPANY allows public airing of negative feedback in a testing phase for precisely the reason Andy outlines. I work for a software company and we DO NOT publish a list of bugs found in QA testing. We send that feedback to our developers to fix it.

Telling customers about a bug that gets fixed and never made it to external release would NEVER be done. Heck we don't even tell customers about bugs we did not fix in some cases if we believe the conditions that cause the bug to appear are sufficiently rare that the average user will never see it. It would serve no useful purpose to do so and simply creates unwarranted negativity for issues that have no impact on the majority of our users. Those users it would affect are informed.

WotC's credibility has been RESTORED by this response, not damaged. WotC has done absolutely nothing deceptive or manipulative in ANY WAY. PERIOD. What they have done is standard operating procedure for ALL companies that conduct QA testing.

This is a faux controversy and is not proof of some evil WotC conspiracy.

Drkfathr1 · Jan 31, 2008

The important fact for me, is that three designers that I like, and whose work I have enjoyed, and whose opinions I trust, like the new game, and found it fun and enjoyable.

Their opinions mean more to me than "random gamer/tester #10's" would anyways.

La Bete · Jan 31, 2008

Maggan said:
"Don't fuel the fire" is my advice to WotC. Just keep quiet, and release the game and then start up the marketing for real.

/M

Alternatively they could be using the "stoke the the nerd rage to great heights" approach to marketing.

It's what I'd do - admittedly only to warm my toes by the outpouring of rage, but hey.

Delta · Jan 31, 2008

I've commented previously about how WOTC has turned its back on the Open Gaming movement ( http://deltasdnd.blogspot.com/2008/01/part-iii-promise-of-ogl.html ). There's been debate in the OGL forum about why WOTC is even bothering to call its new license "OGL" ( http://www.enworld.org/showthread.php?t=215975 ).

This new commentary is the perfect demonstration of the reversal. In 2000 Ryan Dancey wrote this about Open Gaming rules (still on WOTC site today):

With lots of people able to work on them in public, problems with math, with ease of use, of variance from standard forms, etc. should all be improved over time.

Today Andy Collins says the opposite:

I recognize that this statement will generate controversy, but I don't believe it's helpful to anyone to hear negative comments about a game that isn't even finished (much yet published).

So we went from a philosophy of "work on problems in public" in 2000, to a clear-cut "work on problems in secret" that we have here in 2008. I find that to be a rather remarkable about-face.

Finally, you've got what Andy thinks is his coup-de-grace:

"When we run our software through beta-test here, we don't publish a list of the bugs - we fix them."

You do if it's Open Source. ( https://sourceforge.net/ )

tomBitonti · Jan 31, 2008

So ... there is an ethical issue here.

Let's say that a drug company hires a number of companies to test out a new drug.

There are 10 companies. Five have overall positive results, two are negative, and three are inconclusive.

The drug company discontinues the testing at the two companies with negative results.

The remaining results are published. They are an inaccurate reflection of the testing results.

I have no sympathy for Wotc in regards to this issue. They know exactly what they are doing.

I have a little sympathy for the two playtesters. My thanks for the information that they have provided. But they have put themselves in the middle of a marketing process, and need to have an understanding of how that works, and how that reflects on their integrity.

Now, there is a big difference between drugs and RPG testing. A drug company that does what I outlined is probably breaking the law. For RPG playtesting, I'm OK with we realize that the information is a part of a marketing campaign, and include that in our awareness.

ChaosShard · Jan 31, 2008

vagabundo said:
I think it was completely legit of them to ask for anything negative to be directed to them. Remember that only people who asked, and were specifically trusted not to abuse it, where given some leeway to talk about their impressions.

As usual an internet storm in a teacup.

We'll get a 1000 impressions at the end of feb anyway.

This.

Also, if Playtester X drops a laundry list of complaints on a forum, all it will do is cause 'net drama, because that tester may very well (like Andy said) be working off of older material which has already been fixed.

Besides, the whole idea of beta testing (and this is true of the closed beta-tests in video games) is to send those complaints to developers so they can be addressed, not braindumped to rabid fans so they can form a lynch mob.

HP Dreadnought · Jan 31, 2008

What does it matter if playtesters with negative opinions are allowed to share their views on the forum or not. I can't see how that helps anybody. . .

1. The problems if true are likely to be fixed.

2. There is an EXCELLENT chance that you won't agree with the playtester's opinions anyway.

People have this idea that "playtesters" means "people who think exactly as I do." Just think about how many people have differing opinions from you just on the various forums you participate in. Why do you think only the people who agree with you would be selected to playtest.

Letting them air negative views just contributes to general drama for no real positive effect. Unless of course you happen to be one of the people who just enjoys drama. . . of which there are quite a few in the gaming community if discussions surrounding 4E are any indicator.