Public Playtests Should Be Fully Playable

[Sacrosanct]

Playtesters are only getting the full rules early if those rules are written so well that nothing needs to be edited after playtesting.
And what game was written so perfectly that nothing needed fixing after being tested?

Mine.

 

[Reynard]

Playtesters are only getting the full rules early if those rules are written so well that nothing needs to be edited after playtesting.
And what game was written so perfectly that nothing needed fixing after being tested?

Not "full" as in "finished" but "full" as in "covers all the bases."

 

[GMMichael]

. . .
Put out a complete, stand alone playtest document and then give us the time needed to actually run it long enough to give useful feedback.

What I'm hearing is that you think the final release of Cool Name Goes Here should be about 5-6 years off.

 

[Blue]

I know I have mentioned this in a couple threads but I wanted to highlight it:

You can't playtest a system unless the system is complete (ie fully playable). It is one thing to ask the public to playtest a new class or race or whatever within the context of an existing system, but it is inefficient and counter productive to playtest discrete changes inside a system a couple Olathe a time.

To companies looking for playtest feedback from the public:

Put out a complete, stand alone playtest document and then give us the time needed to actually run it long enough to give useful feedback.

Counterpoint: You are using the wrong metrics.

The efficiency that matters to the game companies (and should matter to us as well) is designer efficiency. Expecting designers to design a full, playable system without feedback, and then go and have to rework and redo large swaths of the system once they have the first feedback, including ripple and second order effects, rebalance, again put out a full system (after a long wait because of how many interlocking changes would need to be made against a full system), for a second round of feedback, is just horrible, crazily, ridiculously, and business-unsustainably inefficient. Plus it assumes that players will test everything (high level play?, subsystems X, Y and Z, both solo and in conjunction with each other?) so that it doesn't have rounds of feedback and then have some of the base assumptions challenged and make fixes with big ripple effects again. Oh,a nd that all of these players are willign to spend three hours filling in a complete system survey about every aspect, with equal attention to detail for the whole thing.

Instead, putting out a few systems that hang together, like "try these combat bits - they don't include psionics and have simplified terrain and no mounts, vehicles or starship combat, and only these few pregens and those four monsters we hope the math works out on, but we want to see if our basic encounter subsystem works and runs fast and is fun", then doing it again or again.

In other words, putting out a minimum viable product to get feedback on that before building on it.

And that's for a whole new system, not something building on something like OneD&D is building on 5e where we have a "good enough" framework to test parts.

Also, there's a whole lot more player hours to be had then designer hours.

In software design there used to be a model called waterfall, where you design and build everything up front. It's been depreciated by real developers where the business has allowed for a variety of more flexible development methods designed to keep putting things out and getting feedback. Follow the entrepreneurs - "fail fast".

Valuing player efficiency over designer efficiency causes playtest delays or removes playtests entirely due to unreasonably long development costs, which are in turn due to massive rework efforts and the slow feedback cycles those engender.

 

[EzekielRaiden]

I'd like to see a truly iterative public playtest. I wasn't involved in either of the Pathfinder ones or the D&D Next one. As far as I can tell from public responses, it seems WotC actively did not listen to the community response. I don't know how true that might be of the PF playtests, although it appears for new classes and stuff Paizo seems to listen.

Oh, no, they listened—when it suited them. When it didn't, they listened creatively, shall we say.

For example, because the playtest Warlock and Sorcerer (which were incredibly creative and interesting concepts) didn't get instant supermajority approval, they were scrapped forever and we got the poorly-made, bland, barely-tested final products you see today. Conversely, Expertise Dice were consistently unpopular, but beloved by the designers (AIUI, specifically Mearls), so they went through several iterations before finally getting canned fairly late in the process.

So, it's incorrect to say they didn't listen. They did. They just tended to dig in their heels with several ideas that should have been abandoned much more quickly, and conversely they failed to speak up for and defend ideas that should have gotten more than a single playtest packet. E.g. a big complaint about the Sorcerer was that people felt they were being pigeonholed into being a beefy warrior type, but that was only due to the nature of the example subclass, Dragon Soul. If they had instead responded to the feedback with, "We hear you and understand your concerns, but would like you to seriously consider what we're offering, so we're including Shadow Soul in the next playtest packet as a contrasting example," I think most of the complaints would have disappeared and people would have been fairly enthusiastic about the playtest Sorcerer.

But because every single choice HAD to be either a designer's pet project OR nigh-universally popular, these two awesome and creative options got deleted with extreme prejudice.

So yeah. They listen. I just wish they were (much) better at it.

 

[Blue]

Not if you change how often rests are expected based on changes to the classes or the encounter building rules. Not if spells or class abilities you don't have yet can easily relieve exhaustion. Not if DCs change or monster ACs get tweaked.

ABSOLUTELY for all of those. Because they have a goal of backwards compatibility and if they make changes and it breaks something earlier, they will fix the current change and not go back.

Think of what their goals actually are, not what you imagine a new game's goals are.

Oh, and they have already said in one of the videos they are planning on changing the encounter math.

It's lazy and incomplete and designed to keep eyeballs on more than actually improve design. Just be honest and call it "previews" like year of Dragon prior to the 3E launch, because that is what it really is.

Please provide any support for your statement that they are doing it to be lazy. Now you are ascribing motives to doing it this way. Convince me this is a documented and supported fact, because it sounds a lot like it's your bias talking.

 

[Blue]

I'd like to see a truly iterative public playtest. I wasn't involved in either of the Pathfinder ones or the D&D Next one. As far as I can tell from public responses, it seems WotC actively did not listen to the community response. I don't know how true that might be of the PF playtests, although it appears for new classes and stuff Paizo seems to listen.

There's a whole thread after the OGL survey where people where saying they don't read them that there was a huge outpouring form WotC staffers who actually make the game talking about how they deal with the survey and the number of people and the sheer man-hours per survey they put in.

Is this something specific back with the D&DNext? And if so, why muddy the waters now when we know that's not the case with the OneD&D playtest.

 

[Reynard]

ABSOLUTELY for all of those. Because they have a goal of backwards compatibility and if they make changes and it breaks something earlier, they will fix the current change and not go back.

If that were the goal there would be no playtesting because there would be no changes. Backwards compatibility is the easiest design in the world: change nothing.

But they are changing things, and that means systems interact. Determining how all those systems interact requires having all those systems all operating simultaneously.

People in this thread are acting like this has never happened before and we don't have a model for how this works.

If you want to playtest a game, you need to present the game. It's certainly possible that you can present a completely isolated element-- a class or race or set of spells -- but that isn't what is happening here.

You can't test a subsystem that interacts with a different subsystem you aren't also testing simultaneously.

 

[darjr]

Eh. I get why they wanted to do packets. It's not that hard to play test this way, especially for an evolution of a game. I think it's actually less of a burden.

Also KBP is what? Seven people?

 

[Reynard]

It's not that hard to play test this way, especially for an evolution of a game.

I wonder how we ended up with CR systems that don't work and classes that completely fail and have to be rewritten again and again?