4E playtesting or lack thereof

[Fifth Element]

Originally Posted by amethal

His point is that the skill challenge rules that appeared in the Dungeon Masters Guide are not the rules Massawyrm's group playtested.

No, his point appears to be that the published version of skill challenges was not playtested at all, by anyone. We simply don't know if that is true, but it seems unlikely.

[Dice4Hire]

I, for one am impressed with the playtesters WOTC selected, as I have seen no large amounts of rules posted anywhere or even discussed in detail.

But to the OP, why would the rules playtested need to be the rules printed? And why would WOTC restrict itself to using only playtested rules?

Yes, ina perfect world, it might be best to use material playtested. But what if each and every group of playtesters had totally different advice? Should WOTC then print one version of the rules for each playtest group? I do not htink so. Playtest is suggestions, not commandments engraved on stone tablets.

[amethal]

Originally Posted by Fifth Element

Are we really seriously discussing something asserted on a message board by someone calling himself Trollman?

Yes. Why wouldn't we?

As far as I am aware, Frank Trollman is his real name.

[amethal]

Originally Posted by Fifth Element

No, his point appears to be that the published version of skill challenges was not playtested at all, by anyone. We simply don't know if that is true, but it seems unlikely.

Frank Trollman has explained why he thinks that is the case.

I'd be interested know how come it seems unlikely to you.

[ST]

Originally Posted by amethal

Frank Trollman has explained why he thinks that is the case.

I'd be interested know how come it seems unlikely to you.

Would you be willing to restate his argument in your own words, since you're advancing it? I think that would be helpful, since debating the absent Mr. Trollman based on his, from the best I could tell of reading that thread, going off and flaming people for reasons unclear to me, two months ago, on a different forum, does not seem to be a particularly valuable use of our time. That's the subtext i'm getting here, so I figured I'd make it explicit.

If you could also restate the conclusion you draw from his argument, that'd help too, because I still don't really "get" this thread. (I think it's about how the DMG version of skill challenges kinda bites, and post-morteming to see if that's due to playtesting issues? I can't see the point of discussing the obsoleted mechanics, much less *why* they were broken in the first place, so I'm hoping someone can explain where they're going with that.)

Thanks in advance!

[Mustrum_Ridcully]

Originally Posted by Fifth Element

Are we really seriously discussing something asserted on a message board by someone calling himself Trollman?

The name might be meant ironic.

You know, like 10,000 spoons on a rainy day.

[ExploderWizard]

Originally Posted by Mustrum_Ridcully

The name might be meant ironic.

You know, like 10,000 spoons on a rainy day.

BUT ALL I NEED IS A KNIFE!!!!!!

Can't we also say that the errata'd rules were a free ride (when we've already paid)?

Perhaps the playtest results were the good advice- that they just didn't take.

[Samuel Leming]

Originally Posted by amethal

Yes. Why wouldn't we?

As far as I am aware, Frank Trollman is his real name.

Why? Because Frank has been banned from more RPG websites than I've ever visited. Bringing him up is a good way to elevate tempers.

He often brings up valid points though. Though I'm sure the skill challenge rules were used in playtests and commented on, there's no way they were subjected to an actual analysis. The math for doing that is tedious but simple. It would be nice to know what went wrong.

[Windjammer]

Originally Posted by wedgeski

Classic internet obfuscation. FrankTrollman uses every little trick in the book to raise a riot in that thread, is called on it more than once by the moderator (and threatened with a suspension) and somehow it's Mearls who is in the wrong? Hilarious.

I assume you've read the thread on RPGNet before you made these claims, ok? That assumption being made, I want you to link the posts in that thread to substantiate the bolded part. Please use your own skills in backing up your statements, don't refer to authorities ("the moderator") you then withdraw when convenience suits ("cross-posting from other boards that don't have the same moderation guidelines as ENW"). If I missed something deeply disingenious about Frank's posts, please point those bits in his posts out to me. Thank you.

Originally Posted by wedgeski

, and on top of that insinuating that one of the site's most respected posters is lying or stupid (that's the only conclusion I can draw from your "priceless" comment).

Yes, and by doing so you violate a central rule on Enworld which reqests not to ascribe implicit intentions or motives to other posters.

I've mounted one simple claim - "no single version of skill challenges WotC released in June 2008 to June 2009 was solidly playtested" - and rather than attributing motives to me or patronizingly diagnose me with teen anger I'd kindly ask people to engage with the claim itself. Because you know what? I welcome to stand corrected on this one.

[Fifth Element]

Originally Posted by amethal

Yes. Why wouldn't we?

As far as I am aware, Frank Trollman is his real name.

Just pointing out the irony.

[Fifth Element]

Originally Posted by Windjammer

I've mounted one simple claim - "no single version of skill challenges WotC released in June 2008 to June 2009 was solidly playtested" - and rather than attributing motives to me or patronizingly diagnose me with teen anger I'd kindly ask people to engage with the claim itself. Because you know what? I welcome to stand corrected on this one.

That's not a simple claim, though. It contatins a very nebulous term - "solidly" - that renders it impossible to know exactly what you mean. What sort of playtesting do you consider "solid"?

The wording - "no single version" - also discounts the possibility of solid playtesting of the system as it evolved during playtesting. What do you consider a "single version"?

[Fifth Element]

Originally Posted by amethal

I'd be interested know how come it seems unlikely to you.

Why it seems unlikely that they were not playtested at all, by anyone? Those were my words. I find it unlikely that professional game designers would not playtest something at all, and release it. They get the benefit of the doubt that there was at least some minimal amount of playtesting. That seems reasonable to me. Otherwise we're assuming incompetence, which I don't think is fair.

[jensun]

Originally Posted by Windjammer

I've mounted one simple claim - "no single version of skill challenges WotC released in June 2008 to June 2009 was solidly playtested" - and rather than attributing motives to me or patronizingly diagnose me with teen anger I'd kindly ask people to engage with the claim itself. Because you know what? I welcome to stand corrected on this one.

When asserting such a claim it is traditional to provide some evidence to support it. Beyond some unsubstantiated assertions made by someone else that's something you haven't yet done.

[Dice4Hire]

Originally Posted by Windjammer

I've mounted one simple claim - "no single version of skill challenges WotC released in June 2008 to June 2009 was solidly playtested" - and rather than attributing motives to me or patronizingly diagnose me with teen anger I'd kindly ask people to engage with the claim itself. Because you know what? I welcome to stand corrected on this one.

Please detail and justify your gauntlet with facts and references, and I will be glad to pick it up.

[amethal]

Originally Posted by ST

Would you be willing to restate his argument in your own words, since you're advancing it?

With pleasure!

Massawyrm’s very favourable review of his playtest of 4th edition included a glowing review of skill challenges.

I don’t know at what date he carried out his playtest (and whilst the impression I get is that he wrote about it shortly afterwards, I don’t have a date for his post either). However, at some point after that but before sending the DMG to the printers WotC changed skill challenges.

The rules in the DMG no longer include rolling for NPCs “counter-arguments” and the range of possible results (which Massawyrm seemed to particularly like) was replaced by a simple pass or fail.

Neither Massawyrm nor anybody else, to my knowledge, has posted about playtesting the DMG version. This may be due to the terms of NDAs. Mearls could have clarified the situation if he wanted to, but chose not to. This is all “absence of evidence is not evidence of absence” territory but at least raises the possibility that they didn’t playtest the DMG version.

Clearly, the DMG version was not playtested sufficiently, as it required correcting soon afterwards. I am really not interested in the mathematical side of D&D, but apparently skill challenges were either far too hard or far too easy, depending on the circumstances.

I am very hazy on the details, but the consensus on EnWorld is very much that they needed fixing, and indeed they were corrected by WotC very quickly – apparently in response to the concerns of players.

It is Frank Trollman’s contention that even a cursory playtest would have identified the problems in the DMG version, as they are glaringly obvious. They aren’t that obvious to me, but then I haven’t played through a skill challenge.

Thus either WotC didn’t playtest the skill challenges which appeared in the DMG, or they chose to publish something they knew was wrong, or the playtesters failed to spot something that a great many other people (apparently) spotted straight away.

[Obryn]

Originally Posted by Windjammer

Yes, and by doing so you violate a central rule on Enworld which reqests not to ascribe implicit intentions or motives to other posters.

OK - then what did you mean by the "priceless" thing?

I've mounted one simple claim - "no single version of skill challenges WotC released in June 2008 to June 2009 was solidly playtested" - and rather than attributing motives to me or patronizingly diagnose me with teen anger I'd kindly ask people to engage with the claim itself. Because you know what? I welcome to stand corrected on this one.

Mathematically, skill challenges as presented in the DMG are rough, but not unworkable. That is, you can run skill challenges with the old DCs without your game exploding, and the rules are quite simple to follow. I'm a bit surprised the math wasn't scrutinized more closely, but when it comes down to it, you could still run successful encounters with the final system. If a lot of the concentration was on higher-complexity skill challenges, it's not surprising the playtesters found more success than failure.

Regardless, I don't know that this particular problem is something that playtesting would have caught. Having run a few pre-revision Skill Challenges, they actually do work (as in, they are playable) it's just the success rate is low at certain levels. I think it's better-suited to mathematical analysis, like Stalker0 ran, and that playtesting isn't the right tool to find its flaws.

I also don't know what sorts of revisions the table went through. Were the DC as-published? Was the footnote about skills there originally? I'm honestly not sure.

-O

[Belphanior]

As a quick aside, am I the only person in the entire world expecting new skill challenge rules in DMG2? Because that would seem like a perfect book in which to publish optional system variants and/or expansions.

[amethal]

Originally Posted by Fifth Element

Why it seems unlikely that they were not playtested at all, by anyone? Those were my words. I find it unlikely that professional game designers would not playtest something at all, and release it. They get the benefit of the doubt that there was at least some minimal amount of playtesting. That seems reasonable to me. Otherwise we're assuming incompetence, which I don't think is fair.

Sorry, I got caught up in Frank Trollman's hyperbole. I didn't mean to start arguing for an absolute position.

I'm happy to accept that they had minimal, cursory and utimately inadequate play testing.

And I am certainly not intending to attribute incompetence, or worse, to any game designers. There are lots of pressures in publishing a new game, and mistakes are inevitable.

[Fifth Element]

Originally Posted by amethal

Thus either WotC didn’t playtest the skill challenges which appeared in the DMG, or they chose to publish something they knew was wrong, or the playtesters failed to spot something that a great many other people (apparently) spotted straight away.

Now, this is different than an assertion that #1 is the case. You've offered three options, which I think cover the possibilities pretty well.

To me, #3 seems the most likely. I personally don't have any issues with the system in the DMG, but I know many people do. Maybe the designers just saw it the way I do, who knows.

We don't really have any evidence to support any one of these three possibilities, so if you are going to assert that a particular one is correct, you need to provide the evidence when you do so.

[Nymrohd]

I guess people are practicing their new Forum Troll PrC features?