Who Playtested This Anyway?

The first playtests of Dungeons & Dragons were by Gary Gygax, his kids, and his friends. The industry has evolved considerably since then and playtesting along with it. A new playtesting methodology was borrowed from software development, and it's likely to influence how game companies produce products in the future.

ian-gonzalez-1256899-unsplash.jpg

Photo by Ian Gonzalez on Unsplash
[h=3]The Early Playtests[/h]Gary Gygax, co-creator of Dungeons & Dragons, originally playtested early drafts of D&D with his kids according to David Ewalt:
The first people to play it were Gygax’s eleven-year-old son, Ernie, and nine-year-old daughter, Elise. Gygax had created a counterpart to Arneson’s Blackmoor, which he called Castle Greyhawk, and designed a single level of its dungeons; one night after dinner, he invited the kids to roll up characters and start exploring. Ernie created a wizard and named him Tenser— an anagram for his full name, Ernest. 3 Elise played a cleric called Ahlissa. They wrote down the details of their characters on index cards and entered the dungeon. In the very first room, they discovered and defeated a nest of scorpions; in the second, they fought a gang of kobolds— short subterranean lizard-men. They also found their first treasure, a chest full of copper coins, but it was too heavy to carry. The two adventurers pressed on until nine o’clock, when the Dungeon Master put them to bed. Fatherly duties completed, Gygax returned to his office and designed another level of the dungeons.
The RPG industry didn't exist back then as we know it today, so the limited scope of his early playtests were understandable. He expanded that scope over time to include his friends and colleagues. Things changed once the RPG industry matured. Gygax would often publish early rules in Dragon Magazine as a form of playtest release. In later editions of D&D, this relationship between magazine and game became more formal:
During its second Wizards of the Coast run, the Dragon magazine staff was aligned with the D&D R&D staff. This allowed the magazine to integrate more closely with the actual D&D game than ever before. This began in Dragon 360 with a new “Design & Development” column—which didn’t just preview D&D 4e, but also explained the reasoning behind many of the design decisions, offering a level of interaction with the D&D creators that had never been seen before. Starting with Dragon 365 (July 2008), readers could also playtest upcoming D&D rules—the first of which were drawn from the Eberron Player’s Guide (2009).
That alignment happened around Fourth Edition, which was when D&D started borrowing elements of video game development for its roll-out, including playtesting.
[h=3]The Industry Grows Up[/h]Role-playing games are bound by a basic premise that "anything can be attempted," and thus while there are rules that can be playtested, there are a limitless number of potential unforeseen consequences in a game where players have full agency over their character. Video games are more constrained, although Massive Multi-player Online Role-Playing Games (MMORPGs) have become increasingly complex because of the interactions between masses of players. To that end, video games use open alpha- and beta-stage testing. These playtests release the unfinished game to a limited audience so that the developers can observe and record feedback. Minecraft was an important example of how this works:
An example of a video game that made extensive use of open playtesting is Minecraft, which was made available for purchase in its pre-alpha stages. This both helped to financially support the game and provide feedback and bug testing during its early stages. Playtesting began even before the game features included multiplayer or the ability to save games. Mojang continues to make use of playtesting with Minecraft through weekly development releases, allowing players to experiment with unfinished additions to the game and provide feedback on them.
It's worth noting that playtesting is used as both a form of mass feedback on the game and a marketing opportunity to generate revenue, essentially asking players to pay for the privilege of making the game better in exchange for early access. In fact, the practice has become so commonplace that Game Informer changed its policy to review video games as soon as they begin charging customers instead of waiting until the game is considered "finished." For a particularly egregious example, see Fortnite Battle Royale, which has been in "Early Access" since 2017!

This same approach to playtesting was evident when Wizards of the Coast sold copies of the "Wizards Presents" series for Fourth Edition, which included early previews of the game before it saw the light of day. The playtest for the Fifth Edition, perhaps in reaction to the struggles of Fourth Edition, took on a whole new dimension with an open call that included over 120,000 playtesters. Mike Mearls, senior manager of research and design for D&D at the time, explained the strategy to Fortune:
In some ways we're just catching up to the resources you have in digital games, where you can look at server logs and see what characters are people making, what abilities are they using. It’s kind of us embracing a more modern approach to game design. Especially with something like Dungeons & Dragons, that has such a clear identity and such a huge fan base, it lets us really stay in touch with what people are experiencing. Honestly, without this data and without the feedback we would be designing a much different game.
Just as they did with Fourth Edition, Wizards released a playtest version for sale in Ghosts of Dragonspear Castle. Since then, playtesting has become increasingly a public, data-driven affair that has been used by other large RPG publishers, including Paizo:
In 2008, Paizo launched an unprecedented public playtest aimed at updating the third edition rules to make them more fun, easier to learn, and better able to support thrilling fantasy adventures. More than 40,000 gamers just like you joined in the fun by playtesting the new Pathfinder RPG rules and providing feedback, and the rest is gaming history. Now, 10 years later, it's time to put the lessons of the last decade to use and evolve the game once again. It's time for Pathfinder Second Edition!
Like Wizards, Paizo published print versions of the playtest for purchase:
PDF editions of the Pathfinder Playtest Rulebook, the softcover Pathfinder Playtest Adventure, and the Pathfinder Playtest Flip-Mat Multi-Pack will be available for FREE right here on paizo.com starting August 2. The print edition of the Playtest Rulebook is available in softcover ($29.99), hardcover ($44.99), and deluxe hardcover ($59.99) editions. The print edition of the Playtest Adventure has a suggested retail price of $24.99, and the Flip-Mat Multi-Pack is priced at $24.99.
There are downsides to this transparent approach to game development as White Wolf discovered in a 2017 playtest for the Fifth edition of Vampire. A character named Amelina was described as preying on children -- and the fallout from that playtest (among other issues) led to some serious changes to the company as a result.

The engagement with the audience in shaping role-playing games doesn't end with playtesting though.
[h=3]Night of the Living Games[/h]Mearls explained how feedback would be ongoing as a "living game":
The biggest change affects how we make updates to the game going forward. In the past, we relied on forums, summaries of issues from customer service, and our own experiences with the game to guide the changes we made. Though this approach uncovers parts of the game that people are having issues with, it does a poor job of assessing the magnitude of those issues. The public playtest showed us that we need to cast a much wider net to create a clear picture of what’s going on. To that end, you can expect to see annual surveys that work much like the ones we used to guide the development of fifth edition D&D. These surveys will not only allow us to identify trouble spots in the game, but we can use them to look at how attitudes change over time. By comparing one year’s results to the next, we can gain a sense of how the game is changing.
Plenty of smaller publishers playtest too, of course. Chaosium is notable for including playtest notes in some of its products, so game masters can see how a scenario evolved. Fantasy Flight Games has its own playtesting program, to name a few.

Increasingly, playtesting is no longer limited to groups within a company, but rather open to all who are willing to put in the work of playing the game and sharing their feedback. It's not uncommon to hear the refrain, "who playtested this?" at the table.

These days, it's most likely one of us.
 
Last edited:

log in or register to remove this ad

Michael Tresca

Michael Tresca

Blue

Ravenous Bugblatter Beast of Traal
I actually wish more companies did this, a "liner notes" if you will on the headspace of the designer, so that some of the more ambiguous items in a rule set might be clarified. I admit it can be tough to know if a particular component is going to be troublesome, however.

13th Age does this. Plenty of sidebars abtu why the designers did something one way, and the effects of tweaking and changing things. Also when the designers disagreed and both points of view. Really helps you understand the "why" of the rules, as well as make it much more hackable.
 

log in or register to remove this ad

R Karan

First Post
Putting together a good survey can be hard and will affect the answers you get. Are there any examples (from D&D or PF) available online?
 


Shasarak

Banned
Banned
It does not sit well with me when game companies (analog or digital) charge people to be part of a playtest. I mean, it probably won't ever stop until fans refuse to pay, and I don't know how to get that to happen short of a miracle, but... yeah, paying money to help a company balance its incomplete game has always rubbed me the wrong way. I take more issue with it on things like Steam early access than here.

I dont mind paying to playtest something that I like. How else are you going to get a chance to change things from the inside?
 



lewpuls

Hero
I'm going to have to disagree. Playtesting, often widespread playtesting, was common in board games long before that became common in the video game industry. And when the video game industry playtested, it tested for bugs, for ways that the softrware didn't work as intended, rather than for "fun testing", whether the software (when it worked as intended) was actually engaging or "fun" to play.Testing in video games was by the people within the company, many of whom were intimately involved in the game. That wasn't (and isn't) the way to do "fun testing", with people too close to the game.The tabletop game industry taught the video game industry to do fun testing. I remember talking about this in articles and comments for years; now it's not usually a problem, though you still get video game people who think that metrics is all you need for good playtesting.Of course the open betas and early releases are (often) a way to get people to pay for a product that isn't finished, in effect to get people to pay to do your playtesting! That's quite common.I remember reading once that one of Epic's Gears of War games was playtested 40,000 hours [sic]. That's practically impossible for tabletop games, other than big releases like D&D5 or Pathfinder.
 

dragoner

KosmicRPG.com
Something to remember about playtesting is that it is only as good as those willing to listen, having participated in a few, I have seen issues that have come up in play ignored for various reasons.
 

Bobble

Villager
That alignment happened around Fourth Edition, which was when D&D started borrowing elements of video game development for its roll-out, including playtesting.

And that produced a very craptastic, smelly cow patty. That almost destroyed the line.

I don't think S/W testing procedures are needed. Play testing should be VERY late in the process. Your marketing work should tell you the basic data you need in order to design your game. The 4E crew could not conduct good market research to save their lives. If you cannot design, no amount of play testing is going to help you. DESIGNING the mechanics is NOT the job of testers; See Marc's disaster with Traveller.

If you design well and get a good person to be your Editor you should just need a few rounds of limited playing testing to find the holes in the rules, streamline, clarify and polish. 99% of the job shroud be done BEFORE play testing.
 

aramis erak

Legend
And that produced a very craptastic, smelly cow patty.
I disagree quite strongly with this. As do its thousands of fans.

It plays well. Playtesting provided for a game that runs smoothly, provided you don't fight the system.

It just doesn't feel at all like other versions...
That almost destroyed the line.
This part is true, but not for the reason you imply.

The combination of being so different, not being OGL, 3.X being OGL, and Paizo launching Pathfinder....

PF also did a big public playtest... and made several changes from 3.5... but kept the base user experience, at least until they had their own playerbase.


PF being the old game updated and D&D4E being a new game was a big chunk.

PF continuing to be OGL and allowing 3PP supplements while D&D being closed content with very limited 3PP, was a second chunk.

Playtesting for PF was as much advertising as genuine playtest. For many, like me, it was proof that it wasn't for me. For many others, it was proof that 3.X could be improved and still be familiar.

Open playtesting is a form of advertisement, as well as a means of improving the game.

It worked well for Mongoose, for Modiphius, and for Paizo. The paid open for Star Wars worked really well for FFG.
 

Related Articles

Remove ads

Remove ads

AD6_gamerati_skyscraper

Remove ads

Recent & Upcoming Releases

Top