Do You Think Open Playtests Improve a Game's Development?

[Lanefan]

Yes if done right.

"Done right" means having the system pretty much nailed down before releasing to open playtest, and using the open playtest almost exclusively to find and fix errors, omissions, bugs, glitches, and unforeseen rules interactions.

 

[UngainlyTitan]

Voted that "its complicated.." I do not believe that public playtest are of much benefit in designing a game. I strongly suspect that most responders have not had a chance to actually do any playtesting and even those that do, might only manage a session or two. Not enough time to really stress the system, so the response is based on vibes.
I believe that is a very useful marketing tool particularly if used on conjunction with other market surveys.

If you have the development budget to do 3 out of five things, finding out the least popular two is very valuable.
The exercise is also really valuable in generating hype and engagement in the upcoming product.

 

[FitzTheRuke]

Yes if done right.

"Done right" means having the system pretty much nailed down before releasing to open playtest, and using the open playtest almost exclusively to find and fix errors, omissions, bugs, glitches, and unforeseen rules interactions.

That's about the size of it. They certainly can be used to improve a system - they just generally are used for other purposes. Likely all of the purposes that people suspect are somewhat true: They're used to test for potential backlash and to build hype, more than to improve the game (though I am sure that improvement is never not a part of the goal - it's just not a primary one).

As much as I personally enjoy 2024 as a - slightly - better version of 5e, I think that it would have benefitted from a much later-in-its development playtest (the community certainly caught all of its worst 'holes' in record-time, unfortunately after it went to print).

Though I suspect that most of its flaws come down to it being rushed at the end, as most D&D products are, primarily due to internal conflicts at WotC/Hasbro. If it had a full team with proper scheduling, it would probably have been better, regardless of public playtesting.

 

[Thomas Shey]

I don't necessarily think open playtests improve things. I do think blindtests do.

 

[Emerikol]

I don't think it is even intended to actually be much of a playtest. It's more of an early marketing strategy. They want to see what hits with players and what they hate. So it's mixed

 

[LordEntrails]

No, it's a marketing tool.

It's about hype and buy-in. It's about fostering FOMO (I don't want to miss my chance to make a difference!) and a sense of ownership (I helped design this RPG!).

The first page of posts showed why it's not a good research tool. Though now maybe with LLMs and big data analytics you can pull out some insights at a reasonable cost. But do I really think someone reads through and analyzes tens of thousands of feedback forms?

 

[Joshua Randall]

do I really think someone reads through and analyzes tens of thousands of feedback forms?

No, but dating back to the 1950s, properly written market research surveys with imposed structured data, could be analyzed.

That fact that mankind has lost both the knowledge and will to do market research the right way, kind of explains why most products are so terrible now.

 

[UngainlyTitan]

No, but dating back to the 1950s, properly written market research surveys with imposed structured data, could be analyzed.

That fact that mankind has lost both the knowledge and will to do market research the right way, kind of explains why most products are so terrible now.

How to do market research has not been forgotten the methods are still there is the literature. It is not compatible with the investor classes aims to do quality products in many consumer sectors but to get into the details would involve some political analysis.

 

[TheAlkaizer]

It depends a bit on your design intent. If you're making something for your own pleasure without much consideration for potential players, customers, then it's probably not really required. It definitely helps to iron out details and bugs. But that's something you can do on your own over time by playing it.

Otherwise? Playtests are absolutely invaluable.

If you are designing a product for many people to play. If you have a specific experience in mind, or if you have a specific audience in mind, you absolutely need to do playtests to make the best product possible. I come more from a video game perspective, but I've seen it again and again, it is an incredibly common mistake to think that you understand your market and your players well enough to take decisions in a vacuum. By playtesting, you will learn a ton of whatever you've designed, if it achieves the experience you had in mind, or if your audience enjoys what you've made. You can take different decisions following that, but decisions without data are not really decisions.

Playtests are also necessary for fixing issues. You might have all the right systems and mechanics to create the right experience and you might be pinpoint on what your audience is asking for... but there'll be typo, oversights, involuntary dynamics. Playtesting reveals you the monstruous underbelly of what you've created.

Now, as others have mentioned, there are definitely better ways to run playtests. And we could argue about the value of closed and open playtesting. I think it depends largely on your product, your goals and your means. But decades of cumulated experience in board games, videos games and all type of games have proven time and time again that playtesting is absolutely vital to have decent chances to make an excellent product.

 

[Reynard]

It depends a bit on your design intent. If you're making something for your own pleasure without much consideration for potential players, customers, then it's probably not really required. It definitely helps to iron out details and bugs. But that's something you can do on your own over time by playing it.

Otherwise? Playtests are absolutely invaluable.

This is a question about public playtests specifically.