Playtesting Feedback - How Much Influence Will It Have?

[Olaf the Stout]

In the Escapist article announcing 5E I was very surprised to read that in the development of 4E, the feedback that WotC received from the various 4E playtesters was, on the whole, ignored.

So, in effect, the whole 4E playtesting exercise was just a marketing exercise to get people excited about the new edition. There was no real intention to actually use the feedback to try and improve the rules that they had come up with.

Without trying to be too naive, I was genuinely amazed that:

1. They admitted this fact.
2. That they didn't even bother to see if there were a few common threads to the feedback that might point out spots in the rules that needed to be fixed before the game was released.

Without trying to get too negative about a process that hasn't even begun yet, why should we believe that this time will be any different to last time? I want to believe them when they say that player feedback will be an important part of the development process, but once bitten, twice shy and all that jazz.

So how much influence do you think player playtesting will have on the rules? Do you think that very little will change as a result of feedback, it will have a major impact, or it will be somewhere in between?

Olaf the Stout

 

[Gadget]

To be fair, the guy in the article said he typed up a big (20-40 page?) report and sent it in. It sounded like it was unsolicited, or rather, from what I recall of how they talked about play testing at the time, they were looking for very specific sort of feedback. I. E. How does this ability match up with that ability? What about the number of Healing surges/ Hit points for this class? Did this power seem unbalanced compared with that other one? It seems, to use a computer programming analogy, that they did some unit testing, but did not do a whole lot of integration testing. They weren't looking for feed back on the system as a whole, but a very narrow range or parts of the system.

Though, I could be wrong, I was not a play tester. Maybe they did just go through the motions with a full steam ahead, damn the torpedoes mentality. But the above explanation seems to make sense if one is willing to give wotc the benefit of the doubt.

 

[Olaf the Stout]

I don't think it was unsolicited. I think his group must have been part of the official playtest that was selected via D&D Insider or RPGA membership, since he mentions getting his name in the rulebooks as a playtester.

Mearls comment indicated that basically all feedback was ignored, so it must have been very specific feedback that they were looking for.

Olaf the Stout

 

[foolish_mortals]

there's only one or two ways I can think of to get back people's faith in being listened to and that is releasing rough drafts of the game to the public and show exactly where player feedback has improved the game with each public rough draft release. It's that simple. I don't know why they think they can fool people a second time around. Why would they change their ways? It's still the same people working there under the same managers and it's been that way for years. It's just the new political blabber that's coming out of there.

I guess the second way is just to move the game to a new part of the company with completely different people running it. That's the real step they need to do to give people a sense of breaking away from the recent bad behavior. People are not going to associate the bad experience with just the managment, their most likely to point at the game designers.

The new leaf line might work with people who are pleased with the 4th edition but the Pathfinders will probably stay where they are.

I don't know what else to say about it.
foolish_mortals

 

[I'm A Banana]

Here's the quote for posterity and clarity:

Previous editions of the game had play testing periods, but Wizards restricted access to freelancers or those connected to the company and those tests were ineffectual at best. I was in a play testing group for 4th edition back in 2007, and we submitted a 30 page annotated document of what we felt worked and what didn't work with the rules we played. Other than my name among the hundreds of play testers in the back of the 4th edition Player's Handbook, nothing I submitted made it into print. Our feedback was summarily ignored, and Mearls admitted that was essentially true of all the feedback Wizards received from the 4th edition play test.

I think they had a really clear idea of what they wanted to do with 4e. Which is a problem if it doesn't mesh up with what your audience wants you to do with a 4e.

 

[mkill]

It's still the same people working there under the same managers and it's been that way for years.

You must be new. WotC has fired the entire D&D team twice over since 4th ed came out. Ok, that was an exaggeration, but only slightly. Mike Mearls is in the 4th ed PHB, and he's still around, as an Ascended Fanboy. Monte Cook is back from 3rd ed. I don't think anyone else from the 4th ed credits is still around.

 

[foolish_mortals]

You must be new. WotC has fired the entire D&D team twice over since 4th ed came out. Ok, that was an exaggeration, but only slightly. Mike Mearls is in the 4th ed PHB, and he's still around, as an Ascended Fanboy. Monte Cook is back from 3rd ed. I don't think anyone else from the 4th ed credits is still around.

that's probably going to be the perception for a lot of people. Maybe that's good enough. I still think if you want to change people's perceptions you need to move it to an entirely new crew under new management. Really bring a breath of fresh air and not have people from 3rd or 4th work on it.

foolish_mortals

 

[keterys]

Playtesting is a tricky beast... especially with D&Ders who tend to be extraordinarily opinionated and all think they're closet designers.

Without going into specific details, I can say that when playtesting for WotC my feedback has most certainly been used in many cases. And ignored in some others. But certainly enough that I've felt I had a real impact.

I can also say that one quick way to have a developer ignore your feedback is to send them a small tome trashing their idea and submitting a whole new chapter about how you'd totally do (insert game element). I wouldn't be surprised if WotC got a _lot_ of that for 4E.

But, they also clearly made some changes based on feedback. In fact, I believe that some of the biggest problems with 4e (damage scaling, expertise, etc) might be based on some too quick adjustments made very late in the game, too late to send out for another round of playtesting.

 

[Jan van Leyden]

Here's the quote for posterity and clarity:

...

I think they had a really clear idea of what they wanted to do with 4e. Which is a problem if it doesn't mesh up with what your audience wants you to do with a 4e.

So because their remarks didn't make it into print they know it has been "summarily ignored"? I'm really missing the quote of Mr. Mearls' remark or answer here.

 

[delericho]

I suspect general playtesting will have some impact, but not a lot. With so many voices crying out, and so many competing desires, I suspect it will be hard to extract much that's useful.

However, if they throw something out there and hear a huge "Yes!" or "No!", I daresay they'll act on that.

That said, I wouldn't be at all surprised if they had identified some groups of "favoured playtesters" - groups that they've worked with in the past that have a good record of providing feedback, and also of providing balanced feedback. I expect they'll use those groups pretty extensively, and allow them a much greater ability to shape the new game.