Public Playtests Should Be Fully Playable

[doctorbadwolf]

I'll reiterate because it is important: you cannot test a complex system with only a few disjointed parts.

Yes, you can, when that system runs on a well known chassis and is largely compatible with it.

Gamers "eyeballing" rules is worse than useless. Most gamers are terrible at design.

Playtesters are not designing the game. They are reviewing parts of it and stress testing it.

And again…part of that stress testing is making sure that it is compatible with the options they’ve already published.

You need to test stuff in its full context.

Eventually, but even then, only really when creating a whole new system.

You’ll eventually your full system test. After the things the design team thinks need a smell test get put in front of the community. Because that feedback is more useful than some guys attempt at armchair design.

Otherwise you end up with the 5E CR system.

That definitely is not how you get the 5e CR system. Abandoning a rational system to appeal to grognards is how that happens.

Anyone familiar with 5e can look at the play-test doc KP put out and provide useful feedback and ask useful questions.

 

[Fenris-77]

The 5E CR system is an abomination. I doubt it could be recreated in another system without specific design goals to that effect.

 

[doctorbadwolf]

Software design repeatedly shows this is incorrect. Like, all the freaking time. It's one of the reason we write unit tests, so we can quickly prove that the new parts haven't messed up the old parts.

Yep. We know what the chassis is.

The specific numbers are the easiest thing to predict and to iterate. The designers need us very little for the math. The interactions between systems is the designers’ job. Playtester’s job is to give subjective feedback about whether they want the thing, and to use systems bits and see if anything sucks to use.

In RPG testing mode, you can put out a few pregens and thrown together monsters to test an incomplete combat system even though you don't have character generation, monster generation, encounter balance, or mounts, vehicles, hazards, or a number of other parts of combat done. But you can see if your foundation is good, and then you can add onto it. Same for other parts.

Absolutely.

They’ve probably already recieved heaps of actionable feedback on what they presented. Once they know if people like talents, and how backgrounds are set up, and how races are written, they can either iterate or build on those elements.

And by the time the playtest is a full system that requires no reference to the SRD, the community will be very familiar with all the bits.

 

[Fenris-77]

Well, allow me to disagree at least in part. Not about whether or not the designers can do math (they can, I'm sure) but that the designers have a real handle on what that math means once it hits the table. That part I would suggest needs extensive playtesting no matter how keen the math behind the mechanics.

 

[doctorbadwolf]

Well, allow me to disagree at least in part. Not about whether or not the designers can do math (they can, I'm sure) but that the designers have a real handle on what that math means once it hits the table. That part I would suggest needs extensive playtesting no matter how keen the math behind the mechanics.

Eh extensive depends on the complexity level, but is almost always a matter of brute force stress testing, not actually asking playtesters to understand the underlying math of the game.

“This made combat in this situation go haywire” is useful. Very few playtesters are giving useful advice on how to fix the issue they’ve discovered.

 

[Fenris-77]

Eh extensive depends on the complexity level, but is almost always a matter of brute force stress testing, not actually asking playtesters to understand the underlying math of the game.

“This made combat in this situation go haywire” is useful. Very few playtesters are giving useful advice on how to fix the issue they’ve discovered.

What I'm getting at is that the underlying math represents a design choice that tends to shade a game one way or another regardless of anything else. It's not about breaking the math here, but rather deciding what the math means in terms of what the game actually does (often versus what it is advertised as doing).

 

[doctorbadwolf]

What I'm getting at is that the underlying math represents a design choice that tends to shade a game one way or another regardless of anything else. It's not about breaking the math here, but rather deciding what the math means in terms of what the game actually does (often versus what it is advertised as doing).

Okay.

The designers are who needs to actually deal with that math, regardless. Playtesters don’t fix math. Full stop.

 

[Fenris-77]

Okay.

The designers are who needs to actually deal with that math, regardless. Playtesters don’t fix math. Full stop.

That's true, no doubt. However, there are many kinds of issues that could crop up in playtest that index issues with the underlying math. Generally speaking, the mechanics exists to indicate what kinds of conflict the game is focused on, and how those conflicts rank and will be resolved. There can very much be issues all over that idea. 5E, for example, has a ton of them, but never did the kind of recursive playtesting necessary to figure it out (see my comments about the CR system above)..

 

[doctorbadwolf]

That's true, no doubt. However, there are many kinds of issues that could crop up in playtest that index issues with the underlying math. Generally speaking, the mechanics exists to indicate what kinds of conflict the game is focused on, and how those conflicts rank and will be resolved. There can very much be issues all over that idea. 5E, for example, has a ton of them, but never did the kind of recursive playtesting necessary to figure it out (see my comments about the CR system above)..

Eh the CR just should have been prioritized earlier in the development cycle. It was basically thrown together by feel at the last minute, far as I can tell.

My point, though, is that playtesters don’t need the whole stystem to find problems like that, they just needed to playtest encounter building with CR, and that the role of playtesters is not such that it matters whether they understand the whole system. The designers need to understand it, playtesters need to break it and report where it broke and how.

 

[Fenris-77]

Eh the CR just should have been prioritized earlier in the development cycle. It was basically thrown together by feel at the last minute, far as I can tell.

My point, though, is that playtesters don’t need the whole stystem to find problems like that, they just needed to playtest encounter building with CR, and that the role of playtesters is not such that it matters whether they understand the whole system. The designers need to understand it, playtesters need to break it and report where it broke and how.

So I agree with you, completely. What I'm getting at is that it seems more common than I'd like that the designers don't actually like to adress what might actually be systemic issues with the mechanics.