4E playtesting or lack thereof

[ggroy]

In a post by Windjammer in the closed thread "WotC Strategy of Planned Obsolescence?", an issue of playtesting 4E stuff was brought up.

http://www.enworld.org/forum/4842667-post17.html

In that vein, I recommend reading the same author's observations over there (and especially his two follow-ups here). As he points out time and again, WotC has NOT playtested a single version of skill challenges to date (June 2008-June 2009) and has not released a version that is actually used by the designers in their home games, whether in a core book for which they charged $30 or an similarly pricey online service. Which is disheartening.

Frank's 2006 post seems to suggest "never mind WotC stopping to playtest - it's for 4E", with the clear implication "WotC is directing all its playtesting efforts into 4E (ergo, 4E will be solidly playtested all around)". As it turned out, that was overly optimistic.

Does anyone know if the splats WotC currently releases are solidly playtested? Or are we back at the stage of the 2nd 3.5 Complete Series which was commissioned to be written by freelance writers and had only moderate quality control?

I vaguely recall something about the RPGA being used to playtest 4E before it was released. I remember reading older Dragon magazine articles which mentioned older editions also used the RPGA for playtesting, such as for 2E AD&D.

Does anyone know how much playtesting was actually done on older splatbooks, campaign settings, etc ... back in the 1E/2E AD&D and 3E D&D days?

Some of the links in Windjammer's article refers to articles by FrankTrollman suggesting that very little to no playtesting was actually done on later 3.5E splatbooks, such as "The Book of Nine Swords".

If it turns out that the 4E splatbooks had very little to no playtesting done on them before publication, it would be interesting to see how robust 4E is to overpowering/underpowering and whether balance is determined by a publicly unknown (ie. proprietary WotC) secret mathematical formula run on all new crunch rules.

[Lanefan]

edit: it helps if I read what I'm replying to. Nothing to see here...please move along...

[Mouseferatu]

As he points out time and again, WotC has NOT playtested a single version of skill challenges to date (June 2008-June 2009)

Hogwash. My group had access to numerous incarnations of the playtest rules before 4E's release, and they absolutely had skill challenges included.

[Hunter In Darkness]

Not constructive, never mind

[La Bete]

Hi,

I don't think the mods look too kindly on a person creating a new thread based off their previously locked one.....

[GMforPowergamers]

Originally Posted by Mouseferatu

Hogwash. My group had access to numerous incarnations of the playtest rules before 4E's release, and they absolutely had skill challenges included.

Ari. with out giving away too much (aka break non disclosure stuff) can you give us an idea of what WotC playtest stuff is like...or was like??

1) did you see all classes and races tested?
2) did you see all three teirs tested??
3) Do you know how they test (as an example) all the items in adventures vault?? do they just make characters with each oone?

[Mouseferatu]

Originally Posted by GMforPowergamers

Ari. with out giving away too much (aka break non disclosure stuff) can you give us an idea of what WotC playtest stuff is like...or was like??

1) did you see all classes and races tested?
2) did you see all three teirs tested??
3) Do you know how they test (as an example) all the items in adventures vault?? do they just make characters with each oone?

I had access to pretty much the entire early drafts of the PHB, DMG, and MM. I know at least some other groups also had access to the entire rules set, though there might have been some that were only given parts.

As far as how they test specific items from books like AV, I honestly have no idea. My playtesting experience is limited to the main core material (and at the time, I personally was more concerned with learning the rules for my writing contracts than anything else).

[Baumi]

If it is not playtested, why do we have so many playtest-Articles and several changes from Playtest to the Published Material (Barbarian, Artificer,..)?

Also why is it that there are no stinker or overpowered classes so far if there was no playtests? Where are the Pun Puns of this edition?

[Hunter In Darkness]

Originally Posted by Baumi

Where are the Pun Puns of this edition?

Not to get off track but your need a while longer for that. If I recall pun pun could not be donme without more then just the 3 core books.


Maybe I am recalling wrong but it's the splats that brought many of the worse offenders into being. Well that and GM's who didn't know how to say no it seems

[Plane Sailing]

Originally Posted by Mouseferatu

I had access to pretty much the entire early drafts of the PHB, DMG, and MM. I know at least some other groups also had access to the entire rules set, though there might have been some that were only given parts.

I have been told by some groups that they only had access to some parts; it did seem that there may have been a two-tier process of playtest groups.

It is difficult to judge since the playtesting was more secretive in 4e than it was in 3e (I like the 3e playtest groups 'bits' they had when coming up to the launch of 3e, BTW - it was an interesting insight into the process and the changes that came about because of the playtesters.)

Regards,

[Mustrum_Ridcully]

Originally Posted by Hunter In Darkness

Not to get off track but your need a while longer for that. If I recall pun pun could not be done without more then just the 3 core books.


Maybe I am recalling wrong but it's the splats that brought many of the worse offenders into being. Well that and GM's who didn't know how to say no it seems

I think pun-pun even required 3PP material, but I am not sure. Definitely required a sizable and possibly odd combination of supplements at work.

I wouldn't really care about pun-puns. Who would ever play that in a real game?

What you might need to look out for are:
CoDzillas equivalents. Classes that fill multiple roles equally well without sacrificing in any area. Or classes that fill one role better than all other classes and are no worse off then classes in their role outside their area of expertise.

Orb spell equivalents: Powers that are better than most others of their type by overcoming typical limitations for powers of their type. Orb of Force - no spell resistance, deals damage no one is resistant too, comparable damage to similar level powers.

Cherrypicking: Combining only the best abilities of different sources. In 3.x, you could multiclass with Ranger to get Two-Weapon Fighting for free, or multiclass to barbarian to get Fast Movement and Rage - just add Extra Rage! Or you multiclass to Fighter for the extra feat and armor proficiencies. Or pick your first level in Rogue for extra skills or only the first 2 levels in a Prestige Class to get its best abilities.
4E has closed the obvious ways, but there might be some hidden in multi-classing, paragon paths, epic destinies, hybrid classes, spellscars, dragonmarks and so on.

Scry-Buff-Teleport: Standard tactics (possibly requiring more than one character) to devastate even the toughest opponents. An extreme like Scry-Buff-Teleport might no longer be possible due to various changes in 4E, but there could still be the "ideal" combination of powers used. I think the various stun-lock scenarios against Solos are already good examples for this. The real trick is finding a scenario that doesn't break down due to a bad attack roll at the beginning. (But I would not be surprised if some scenarios already exist.)

[Windjammer]

Well, this is priceless for Ari Marmell to come into this thread and say that skill challenges were solidly playtested. By the way, before we get into arguments, let's stick to precision. What I said, paraphrasing Frank Trollman, was roughly this:

Not a single version of skill challenges that WotC RELEASED in June 2008-June 2009 was solidly playtested.

Enter Ari: "Hogwash".

Well, basically, no it isn't Hogwash when the DM in Ari's group, Massawyrm, himself demonstrated - for everyone to see - that Ari didn't playtest a version that WotC ever released. Here's the proof (from the same page I linked in the post mentioned in the OP):

Originally Posted by Frank Trollman on RPGNet

Remember when Massawyrm gave that glowing review at the beginning of this edition?

or his description of skill challenges?

Right. That's what they promised. What did they deliver? They delivered a system that not only is a binary pass/fail, but also didn't even work. Like mathematically, it was unplayable. Despite the fact that they guaranteed us repeatedly and in so many words that "everything just works" in 4e, this was not true. And more importantly, it was obviously not true. Anyone who played through a skill challenge even once would have seen that immediately.

So either they sincerely believed that rules that they had never tested or played with would magically work perfectly or they rushed something to print that they knew full well was completely nonfunctional expecting to patch it later with something that worked at all. I don't even know which option is worse, but I do know that neither really fills me with confidence when they say that they are tinkering with it and they'll have something that works properly "real soon."

So yeah: when mearls tries to play it off as hypocrisy on my part that I simultaneously lambaste him for both his product and his methodology, I remain unmoved. The methodology created the product. Frankly, I'm being generous by chalking it up to the effects of an insular design bubble on a complete lack of proofreading or meaningful playtesting. The alternative is deliberate deception coupled with a callous disregard for their consumers.

-Frank

I've bolded the final line because after seeing Mike Mearls' behaviour on that RPGnet-thread (going for Frank's "ego" instead of the issues that he, along others, raised in the thread) and listening to his recent podcast I'll be hugely surprised if WotC really intends to publish a fully playtested version of skill challenges. Ever.

[DevoutlyApathetic]

Originally Posted by Mustrum_Ridcully

I think pun-pun even required 3PP material, but I am not sure. Definitely required a sizable and possibly odd combination of supplements at work.

Pun-Pun really requires only two things; Assume Supernatural Ability and Sarrukh to exist. His chief tricks are all contingent of changing into monsters and using abilities created for monsters on himself.

4E fixed that issue by drawing a fairly bold line between "Player" stuff and "Monster" stuff.

[wedgeski]

Originally Posted by Windjammer

I've bolded the final line because after seeing Mike Mearls' behaviour on that RPGnet-thread (going for Frank's "ego" instead of the issues that he, along others, raised in the thread) and listening to his recent podcast I'll be hugely surprised if WotC really intends to publish a fully playtested version of skill challenges. Ever.

Classic internet obfuscation. FrankTrollman uses every little trick in the book to raise a riot in that thread, is called on it more than once by the moderator (and threatened with a suspension) and somehow it's Mearls who is in the wrong? Hilarious.

You're not going to make any friends here by resurrecting arguments made in locked threads, cross-posting from other boards that don't have the same moderation guidelines as ENW, and on top of that insinuating that one of the site's most respected posters is lying or stupid (that's the only conclusion I can draw from your "priceless" comment).

[amethal]

Originally Posted by DevoutlyApathetic

Pun-Pun really requires only two things; Assume Supernatural Ability and Sarrukh to exist.

To expand on this a bit.

I don't even know where Assume Supernatural Ability is from - how could anybody think that was a good idea! - but as far as I can tell it just means Pun Pun destroys the game much sooner, instead of needing to wait for Shapechange.

Sarrukh are about as obscure as you can get, a Forgotten Realms progenitor race that just happens to have a few members still around in the modern day. Even if you play in the Realms, and have Serpent Kingdoms, the chances of a low level character even hearing a legend about the Sarrukh is practically zero.

As others have said, Pun Pun is a brilliant thought experiment, but not an issue in "real" games.

The way some spells are very easy to abuse, and the way in which 3.5 actually changed some spells to make them more broken, is in my opinion, a much more important criticism of the playtesting of 3rd edition.

[amethal]

Originally Posted by wedgeski

Classic internet obfuscation. FrankTrollman uses every little trick in the book to raise a riot in that thread, is called on it more than once by the moderator (and threatened with a suspension) and somehow it's Mearls who is in the wrong? Hilarious.

So what? I'd prefer to leave Frank Trollman's unique posting style out of any discussion

His point is that the skill challenge rules that appeared in the Dungeon Masters Guide are not the rules Massawyrm's group playtested.

We don't know if other groups playtested the actual rules. I don't know how playtests work, so I don't know what the "standard" procedure would be.

The fact that they errata'd the skill challenges almost immediately is probably significant here.

The assertion that anyone who played through one skill challenge would spot the flaw is false; there have been people posting here who use the un-errata'd version without apparently finding anything wrong with them. However, these are perhaps not the types of groups who get asked to do playtests.

Personally, I'd love a (presumably short) book on skill challenges, incorporating all the errata and the skill challenge advice that has been given since by Mike Mearls. I think that would be a great addition to my 3.5 game.

[Mallus]

A few words before the lock...

Neither I, nor any member of my group, participated in the 4e playtests.

We have, however, been enjoying a successful 4e campaign since last September.

We find 4e to be the most mechanically balanced edition of D&D to date.

Whether that's due to thorough playtesting, raw talent, or divine intercession, I cannot say.

Are there issues? Of course. Skill challenges needed an errata (and Stalker0's Obsidian system is a nice alternative to have).

All complex systems have bugs. Trust me. I program for a living.

A thing to keep in mind: it can be hard to find problems in buggy RPG rules because they're 'run' on a group of human beings. Human beings are, in this respect, at least, very forgiving pieces of hardware.

Lastly, consider the prior edition. Some core classes were deeply imbalanced (ahem, CoDzilla), multiclassed spell-casters didn't work, the basic bonus-stacking mechanics became unwieldy...

... and yet, 3e was a huge success, still spinning off spin-offs and being enjoyed today.

So what point about playtesting was being made here?

[ST]

What confuses me the most in this thread is the logical leap from "Skill Challenge rules were reworked multiple times" (which is well established) to "4e as a whole was not playtested" (which is pure conjecture), with no rationale linking A to B. I don't find the argument offensive or incendiary so much as, well, not much of an argument.

Similarly, Windjammer, you seem to have a strong opinion but all I can really tell is that you're angry at somebody about something. You have every right to feel that way -- no one can tell you how to handle your internal emotional state -- but as a point of constructive criticism, it's overshadowing whatever point you wish to make.

[ExploderWizard]

I am not privy to any playtest data and I won't accuse WOTC of not playtesting something that they said was playtested.

Since the skill challenge rules as presented in the DMG were playtested before publication I believe it is fair to ask what results came out of those playtests that indicated that the rules were ready for publication?

How much playtest time was devoted to the version of the rules that saw publication?

The math problems showed up fairly quickly in play after release and I believe a large number of customers are curious as to why such issues never appeared in playtest.

It is wrong to automatically accuse a company of lying but if a company is telling the truth then answering such questions shouldn't be an issue.

[Fifth Element]

Are we really seriously discussing something asserted on a message board by someone calling himself Trollman?