Public Playtests Should Be Fully Playable

[darjr]

Never mind.

 

[Bill Zebub]

If that were the goal there would be no playtesting because there would be no changes. Backwards compatibility is the easiest design in the world: change nothing.

But they are changing things, and that means systems interact. Determining how all those systems interact requires having all those systems all operating simultaneously.

People in this thread are acting like this has never happened before and we don't have a model for how this works.

If you want to playtest a game, you need to present the game. It's certainly possible that you can present a completely isolated element-- a class or race or set of spells -- but that isn't what is happening here.

You can't test a subsystem that interacts with a different subsystem you aren't also testing simultaneously.

You're absolutely right that the packet approach is an imperfect playtest, and maybe it shouldn't even be called a playtest but rather a "public commenting period". But it's either that or nothing.

And you can bet your bottom dollar that there's a large group of playtesters under NDA who are seeing material way before we do.

 

[Blue]

You can't test a subsystem that interacts with a different subsystem you aren't also testing simultaneously.

Software design repeatedly shows this is incorrect. Like, all the freaking time. It's one of the reason we write unit tests, so we can quickly prove that the new parts haven't messed up the old parts.

In RPG testing mode, you can put out a few pregens and thrown together monsters to test an incomplete combat system even though you don't have character generation, monster generation, encounter balance, or mounts, vehicles, hazards, or a number of other parts of combat done. But you can see if your foundation is good, and then you can add onto it. Same for other parts.

 

[Reynard]

To pull this away from Kobold and 1D&D a little bit, the purpose of a playtest is to see how parts interact. Game design is art and science. It is inherently iterative. There are certainly parts you can do in isolation -- core mechanics, as an example -- but the farther along you get the more broad the playtest needs to be. And (trad) RPGs are probably the hardest types of games to playtest because they inherently lean on GMs. That's hard to quantify, and equally difficult to account for in design.

All that is why it is relatively rare to see truly innovative design in RPGs. Most of the time, someone is iterating on someone else's work. And, chances are, designing by feel more than anything else. The 80s is littered with the corpses of games that were conceived of in isolation and never tested.

So I am not saying public playtest is bad. I am saying that if the goal really is a game that works over the time scales RPGs are typically intended to work, longer and more complete playtests are necessary.

 

[Reynard]

Software design repeatedly shows this is incorrect. Like, all the freaking time. It's one of the reason we write unit tests, so we can quickly prove that the new parts haven't messed up the old parts.

In RPG testing mode, you can put out a few pregens and thrown together monsters to test an incomplete combat system even though you don't have character generation, monster generation, encounter balance, or mounts, vehicles, hazards, or a number of other parts of combat done. But you can see if your foundation is good, and then you can add onto it. Same for other parts.

Are you sure software design and RPG game design are equivalent?

 

[darjr]

You can see how parts interact by being given the parts that changed. How is that not clear?

 

[AbdulAlhazred]

Here's the problem with this entire line of discussion. If I have designed my entire game, all the interlocking features and design elements, and THEN I give it to you playtest, its TO LATE. Now you just have the PF1 playtest, its nothing but marketing. No game designer is going back and ripping up ALL that work after its near finished. You need to playtest BEFORE the thing is entirely designed.

I mean, I do get that it may not be possible to say exactly how something is going to work in relation to parts of the game that don't exist yet, but you can still try stuff out!

 

[Blue]

Are you sure software design and RPG game design are equivalent?

No, but they have similarities. We can learn from one devotion and see what lessons fit elsewhere.

Waterfall design - designing everything up front and at once, has repeatably been shown not to be an efficient pattern in either time spent or getting what you actually want. It may not be a perfect 1:1 with RPG design, but that doesn't mean that there's no overlap.

Are you sure that there is no overlap with software design and RPG game design such that the concepts can not work?

 

[Fenris-77]

They are one PDF in, I'm not ready to poop in their cornflakes just yet.

 

[AbdulAlhazred]

Anyway, I would just say this much, if Wolfgang Bauer (Kobold Press) decided to do a playtest a certain way, that's the best way. I was a contributor on some of his stuff way back when. The guy is 1000x smarter about collaborative work than all of EnWorld put together. Anyway, I'm highly confident that any PT he's come up with is valuable.