Do You Think Open Playtests Improve a Game's Development?

[mamba]

I also think that most gamers don't know what they want and aren't very good game designers, so listening them has no guarantee of producing better results.

that is what Henry Ford said “If I had asked people what they wanted, they would have said faster horses.”

What do you think?

at a minimum I am not convinced that it helps even though at some level I feel it should, when done properly. I am however convinced that 2024 was done very wrong when it comes to identifying improvements

No clear metrics, no clear predictable results / outcomes based on the feedback. It was barely better than just tossing a coin, both on the side of WotC and the person filling out the survey.

The one thing it probably did was prevent complete brainfarts making it into the finished product, and I assume that is enough for WotC to make it worthwhile. It is not about improving the game so much as about preventing another misfire like 4e

 

[schneeland]

My impression is that open play tests are often a means of marketing rather than the design. I do think that you can get value out of them, too, but that's mostly in the later stages when you want to iron out the last kinks in your design. If you need general feedback on your design in earlier stages, I feel that open play tests are a rather costly means of achieving that and you are usually better served with closed play tests.

 

[Joshua Randall]

2024 5E playtest, in that every time it looked liek they were going to do something daring, i got excited. then they pulled back

You have summed up 5e (both versions) succinctly.

that is what Henry Ford said “If I had asked people what they wanted, they would have said faster horses.”

Henry Ford's quote is funny but misleading. When he asked people what they wanted, and they said faster horses, that's because normal people are not crazy visionary technologists. Henry Ford got exactly the feedback he needed, which was people wanted to get places faster. Thus validating that his product was viable because it met that need.

It is not about improving the game so much as about preventing another misfire like 4e

Bruh. Seriously. What is wrong with you. Keep your weaksauce edition warring out of this.

 

[Nikosandros]

I don't think that they really help development, per se. However, IMO, they are good "market research" and also also provide publicity and engagement.

 

[BrokenTwin]

I was originally on the side of open playtests, but then I remembered just how excited I was at the ideas present during early D&D NEXT that slowly disappeared under the weight of trying to appeal to everybody else's visions.

 

[billd91]

The answer, as I see it, is definitely not No. An open play test helps keep designers working toward something the market wants - which is kind of a big deal for established games and brands. But for new games and small markets, an open play test is less important for development.

 

[RenleyRenfield]

Open playtesting has become common in the TTRPG industry. I think it started in earnest with Pathfinder 1E's playtest, but I might have missed something earlier.

Anyway, I am not convinced that open playtests are actually beneficial to design and development. Especially when those playtests are rolled out slowly and in pieces (like the 5E playtests) I think that the community responses can smother changes and mechanics before people see them in a larger context or over a longer play time. I also think that most gamers don't know what they want and aren't very good game designers, so listening them has no guarantee of producing better results.

What do you think?

Dang, I hope they do!

My game goes out to open playtest later this year (we are still doing internal testing).

But, I dunno how any other companies do it.

We are going to be sending out specific criteria for playtest along with request for feedback on that criteria. So it should make things easier and more succinct for people to playtest since they are asked to focus on specific things.

For example: we might send out something to the effect of "Please create characters and use only X system of magic" let us know if the spells work, break, useless, confusing, overlap with other magic, etc etc" = like that specific. They can then run and play the game however they want, and the feed back they give is just on what we asked so its an easier thing to report back.

They can send feed back on any other part too, that's fine.

We might also do more simple feedback like = "How long did character creation take? What was confusing? What parts did you have trouble looking up? Could anyone not create the character they wanted to? Once you started play, what did you have to go back and fix or add to the character?" = so again, specific "quality of life" for players testing and stuff too...

The idea is to make it so that playtesters can do the thing, and give feedback in less than 4 hours of effort/play.

 

[James Gasik]

I don't mind transparency during a playtest. I definitely prefer knowing what the developers are up to, and the decisions that are being made. I'd love to see actual playtests in action, make videos of them, share stories about "oops" moments.

And to an extent, sure, if they want feedback from the fans, there's nothing wrong with that either- to an extent. But I think you need to stay true to your vision, and set some things in stone right away. Will it mean that some people won't like your product? Yes, quite likely.

But at the end of the day, not everyone was going to like it anyways. Even now, you got a major game company that seems happy wooing 70% or so of their market, after all.

 

[mamba]

Henry Ford's quote is funny but misleading. When he asked people what they wanted, and they said faster horses, that's because normal people are not crazy visionary technologists. Henry Ford got exactly the feedback he needed

he did not actually ask because he saw no value in asking

 

[dbolack]

I have never seen an open playtest actually do that formative/summative research properly.

You have a very limited perspective on value that can be gained. Or perhaps are only interested in a very limited subset of values. That's fair. But that isn't what everyone is looking for and not always what is found.