Playtesting Feedback - How Much Influence Will It Have?

[Rechan]

I think they had a really clear idea of what they wanted to do with 4e. Which is a problem if it doesn't mesh up with what your audience wants you to do with a 4e.

A lot like PF's playtest.

If you had a problem with an area that Paizo did not feel was problematic, then it wasn't going to get changed, regardless of your feedback. It was intended to be 3.75 - ironing out a few fixes for people who liked 3.5, as opposed to addressing grievances.

 

[Raith5]

I love (or is that loved?) 4th ed but I am pretty surpised by the article and what keterys said. I mean if you do the work to set up a playtest and not listen (or have time to listen) to the feedback, what is the freaking point.

The reassuring thing this time is that they have set plenty of time to test everything.

I guess the thing is that if there is outcry (say 60-70% of respondents) concerned about something there is more likely to be some response than if 10% dislike something.

 

[Olaf the Stout]

Playtesting is a tricky beast... especially with D&Ders who tend to be extraordinarily opinionated and all think they're closet designers.

Without going into specific details, I can say that when playtesting for WotC my feedback has most certainly been used in many cases. And ignored in some others. But certainly enough that I've felt I had a real impact.

I can also say that one quick way to have a developer ignore your feedback is to send them a small tome trashing their idea and submitting a whole new chapter about how you'd totally do (insert game element). I wouldn't be surprised if WotC got a _lot_ of that for 4E.

But, they also clearly made some changes based on feedback. In fact, I believe that some of the biggest problems with 4e (damage scaling, expertise, etc) might be based on some too quick adjustments made very late in the game, too late to send out for another round of playtesting.

Oh, I can imagine that it is quite a difficult process. At work last year we put out a government strategic document. Some sections of the document only had a few key stakeholders involved in the development process, yet still took weeks of back and forth consultation to reach a final landing point. Even then there were many areas where some stakeholders weren't happy with the final result.

You're never going to make everyone happy. I wouldn't have given it much thought at all if it wasn't for Greg Tito's quote from his Escapist article

Our feedback was summarily ignored, and Mearls admitted that was essentially true of all the feedback Wizards received from the 4th edition play test.

I can understand that feedback from one group will be directly opposed to other feedback in some cases (i.e. one group complains that PC's have too many hit points, another group complains that PC's don't have enough). In other cases the feedback may be discussing how the game "feels" (i.e. one group may feel that the game isn't gritty enough but the designers aren't going for gritty, so they ignore that feedback).

However, surely there must have been a reasonable amount of feedback from which changes could have been made. Unless Mike Mearls has been taken out of context, the article indicates that the 4E designers ignored basically all of the playtest feedback and just went ahead with their original plans.

I really hope that they have learnt from the 4E experience and are a little more open with the 5E playtest feedback that they receive. That is what I am hoping for at least.

Olaf the Stout

 

[I'm A Banana]

So because their remarks didn't make it into print they know it has been "summarily ignored"? I'm really missing the quote of Mr. Mearls' remark or answer here.

Well, here's the article saying it...again....

Our feedback was summarily ignored, and Mearls admitted that was essentially true of all the feedback Wizards received from the 4th edition play test

If all the feedback -- from every playtest -- is summarily ignored, then the play test isn't really there to help design a better game. That's what playtesters are there for, normally. Sounds like a deliberate marginalization to me.

A lot like PF's playtest.

If you had a problem with an area that Paizo did not feel was problematic, then it wasn't going to get changed, regardless of your feedback. It was intended to be 3.75 - ironing out a few fixes for people who liked 3.5, as opposed to addressing grievances.

Wat.

Okay, that's not at all like PF's open beta, or like the open playtest they're doing now. There's an immense gulf between ignoring all playtester feedback on the one hand, and having clear and immutable design goals on the other. The former disregards any input from the public, which is apparently what WotC did with 4e. The latter only disregards feedback that doesn't mesh with the greater goals, which is actually smart design, since you don't want a mess of a game that is continually recycled and never published since people always want different things. You need goals for the design, and you need specific things you're looking for in feedback. If WotC ignored all the 4e playtesting feedback, then they were looking for exactly nothing, which means that the playtesting was essentially pointless -- playtesting's job is to give constructive feedback.

To equate that complete ignorance of outsider input to Paizo's "We want this game to be mostly compatible with your current game" design goals seems very disingenuous, or at least acutely unaware of what the function of playtesting is in game design.

 

[Rechan]

I'm sorry, [MENTION=2067]Kamikaze Midget[/MENTION], I misread.

But I have a serious problem believing that they disregarded all feedback. I mean if that's so, then how is Keterys' experience possible? If Mearls actually said this, then he should have quoted him.

 

[I'm A Banana]

I'm sorry, [MENTION=2067]Kamikaze Midget[/MENTION] , I misread.

But I have a serious problem believing that they disregarded all feedback. I mean if that's so, then how is Keterys' experience possible? If Mearls actually said this, then he should have quoted him.

No problem!

FWIW, I think the line is a little suspect, too, given the preceeding "We sent 'em 40 pages of house rules!" paragraph, and, well, the function of playtesting. It also doesn't mesh with Keterys's experience, so perhaps Greg Tito is the one being disingenuous here.

 

[keterys]

Well, I don't know what happened with Greg's playtest or what he and Mearls discussed. I don't want to imply anything about someone else's experience.

I'm really iffy on how much I can actually talk about playtesting due to NDA and such, but, if I look at the recent Mord's book, for example, and go through my feedback line by line, they updated dozens of things that I reported as suspect. It made me feel really good - even if they did ignore me on a a bunch of things as well, and I don't like one of the ways they redesigned something So, I could focus on the bad side, or the good side, but either way the process is doing something.

Anyhow, expect WotC to listen to our responses. The trick is, expect them to listen to folks who don't agree with you. And their own internal guys.

I remember when PF originally did its open beta, I objected on a bunch of things (many of which in hindsight were addressed), but I eventually gave up because their change intentions were too limited for me (a bunch of the spells still really bug me, frex). Giving up on joining the process, assuming I wouldn't be heard, means the final product is less useful to me.

So participate. Let's make this the best D&D we can.

 

[Jan van Leyden]

Well, here's the article saying it...again....

Our feedback was summarily ignored, and Mearls admitted that was essentially true of all the feedback Wizards received from the 4th edition play test

So Greg Tito told that Mike Mearls told him that...

If all the feedback -- from every playtest -- is summarily ignored, then the play test isn't really there to help design a better game. That's what playtesters are there for, normally. Sounds like a deliberate marginalization to me.

You're right that would have been a commercial or advertisment, not a play test. I just hesitate to jump to this conclusion given the facts.

I'm pretty sure that the Next Iteration play test will be handled in a different, hopefully better way; at least the powers that be seem to imply this. Nevertheless I fear that some people might think the fact that they can play test and give their experiences, hints, and proposals to the design team will lead to an outcry: "My play test comments were completely ignored. Why did they call this a play test at all?"

Oh well, time will tell...

 

[Insight]

There's a big difference between saying you want feedback (and inviting people to playtest) and actually using said feedback. The mere invitation does not imply that WotC will use any of the feedback.

Sometimes, it's more important for the buying public to have a perception that you care about their feedback than it is to actually employ said feedback.

Now, I'm not stating that this is what's going on, but I am a tad cynical about how much impact the playtesting will have. And this is coming from someone who signed up for the playtest.

EDIT: One thing that would really help the cynical among us to believe in this effort would be if WotC provided some sort of updates as this goes along, especially regarding any changes made directly or indirectly due to playtest feedback. Even without getting into specifics.

 

[pogre]

We playtested 4e and I did not feel that way at all. They were definitely looking for specific feedback about specific game mechanics. A lot of those were adjusted in the published game. Whether from feedback or not I cannot say for sure, but it seems likely.

I'm sure "advice" outside of the official queries was thrown in the bin.