4E playtesting or lack thereof

[Cadfan]

Cadfan, in all seriousness, there's a world of a difference between material like the Artificer or the Barbarian, classes which become part of 4E as soon as their playtest versions are released on DDI, and stuff like hybrid-classes which are banned from RPGA play for a reason - their being half-baked and having not nearly seen enough playtesting for WotC to say "yep, we're fine with it, some tweaking left to do, but nothing that will cause trouble".

But there's no difference at all for the purposes of my post, which is about the logical shiftiness going on here:

Regardless of context, it is ridiculous to claim that no playtesting occurred based on changes made after playtesting. Defining the only playtesting you will accept as being playtesting of "the final version" is just a silly little dodge. It enables the speaker to almost always claim that no "real" playtesting occurred because playtesting almost always leads to changes and tweaks, therefore rendering the playtested version "not final."

And for a sufficiently bitter person, if playtesting occurs which leads to no changes at all, therefore satisfying their demand that the final version be the playtested version, they can claim that the playtesting was pointless because it was obviously ignored.

 

[Mallus]

Yes.

Does 4e seem badly playtested to you?

Bonus questions: did you play prior editions of D&D? Did they seem badly playtested to you?

 

[malraux]

I also suspect that the multiple versions of skill challenges result not from refinements of the mathematical elements, but from changes in what skill challenges are perceived to do. Consider: Should SC be on level with most encounters (in that most parties will likely win) or should SC be something with a higher risk/reward? That changes what you want the math to do quite a bit.

Should SC be resolved quickly, or be part of an ongoing multisession event? I'm sure there are multiple other dimensions SC could be analyzed on as well. The point is that these aren't necessarily questions that can be answered via play testing. Its an overall design question. More importantly, there aren't guaranteed to be right answers.

To me SCs look like they underwent several design goal changes, rather than poor playtesting or mathematical analysis. In addition, after being released to the public, I think the feedback was that many players saw the goal of SCs as different from what the designers predicted.

 

[catsclaw227]

WindJammer -- I think it's tough to look at the ball and not see the man too. The Frank Trollman post wasn't the best example for your cause and I still don't know what the goal of his post was... Was it to call out WOTC designers? Was it to express his concern with the playtesting of skill challenges? Was it to stick a wrench into the gaming enjoyment of the 4e players that are just fine with the skill challenges and/or the errata presented?

Pointing to his post as your evidence (or maybe it was for collaboration on a thought you already had?) only emphasized the potential for this to be a flame/editionwar/complain thread.

I agree that the skill challenges as presented in the DMG needed some work. We ran them a few times prior to the errata and, though they seemed tough, the game fun didn't diminish by their inclusion. Actually, the game was better for it.

Skill Challenges are the right idea. The implementation is proving to be more esoteric than I think anyone could have guessed. Between all the Ruling Skill Challenges articles, forum posts that describe examples in play *of which there are MANY good examples here on EnWorld*, the alternate systems proposed (Stalker0), and the individuality of the DMs that acutally use them, there appears to have been many observations about what works for any particular game table and what doesn't.

How I apply challenge complexity, difficulty (low, medium, high), circumstance bonuses and other modifiers, and how I percieve a success or fail all play into this system. There are a lot of assumptions that can change whether the base skill challenge system is balanced or not. Is this a problem with the core system, maybe, but its much more an issue around how the game is played at the table.

Was it your impression that the released skill challenge mechanics caused widespread problems?

I agree they were mathematically wacky, and that this wackiness should have been caught. They don't look like they're broken at first glance, but a thorough breakdown shows that they don't do quite what you'd expect them to do. But do you believe peoples' home games were negatively affected by the rules as released to an extent that a gaming group not currently engaged in statistical analysis would even notice?

My contention is that the rules are, indeed, broken, but not in such a way that a gaming group - including a playtest group - would necessarily notice.

At my table, we've had no problems with them. The players love the way they work, they all feel involved, failure isn't a terrible thing (as it's usually just a stumbling block), and that it gives a much better experience than "roll once at a DC, problem solved." At my table, they are embedded in roleplaying challenges and have been short or long in game sequences. We recently ran a skill challenge that took up 3 hours of game time, as they gathered clues around a city. It was went really well for all, and I got props after the game.

As Obryn stated above, the playtesting itself could have been a factor if the playtesters agreed with the assumptions the designers had.

Do you think that OD&D or 1E had the level of playtesting that went into 4e? I doubt it, seriously. Yet no one is writing threads like this about their favorite old school game.

This isn't life or death and a math fix for the skill challenge system won't hurt anyone. Nor has it impacted the games I play or run. Has the skill challenge system math caused you to give up on 4e?

Is the point of this thread to state that a complex game has a subsystem that didn't quite work as well with 1 million players as it did with 500 playtesters? Don't all games suffer from the same problems?

Or was it to state without any proof that WOTC never tested skill challenges, neener, neener, neener, how-do-you-like-them-apples?

 

[ggroy]

Does 4e seem badly playtested to you?

Bonus questions: did you play prior editions of D&D? Did they seem badly playtested to you?

4E combat doesn't appear to be excessively overpowered or underpowered, at least for most of the stuff in PHB1. I haven't got around yet to playing any classes or races from PHB2. I haven't used any of the crunch from the Martial Power book yet. The stuff in Arcane Power for wizards doesn't seem to be overpowered or underpowered at this point. Though I haven't tried out every single new thing for the wizard yet. For the stuff I've tried so far, it appears to be balanced.

For prior editions, 1E AD&D seemed to have clunky rules for psionics in the PHB. They didn't seem to be well thought out. No idea how well they playtested it.

I do remember 1E AD&D magic users being very powerful at high levels. Though whether that was a deliberate design goal or a flaw, I don't really know.

For 3.5E, I never got the opportunity to create any munchkin overpowered characters. The DMs I played with banned almost all splatbooks and mainly allowed the core books only. Most of the 3E/3.5E campaigns I played in, the DM ended the game at around level 8 or 9. For the stuff we played (ie. lower than level 10), there didn't seem to be any major overpowering issues I can recall. I can't comment on higher levels, since I never got the opportunity to play any. I felt that the 3E core rules were probably reasonably well playtested for lower levels (ie. less than 10), though this is probably not saying much.

 

[AllisterH]

I think one of the reasons why skill challenges playtesting might not be caught is that this is one of the cases where even if the PCs fail the challenge, the actual challenge itself is a success for the players involved and (most importantly), even though the skill challenge itself wasn't successful, the outcome isn't a game ender.


Reading the comments about skill challenges (both good and bad) over the past several months, one of the common traits is that a good or bad skill challenge seems to have absolutely nothing to do with whether or not the players actually completed the SC successfully.

 

[Windjammer]

WindJammer -- I think it's tough to look at the ball and not see the man too. The Frank Trollman post wasn't the best example for your cause and I still don't know what the goal of his post was... Was it to call out WOTC designers? Was it to express his concern with the playtesting of skill challenges? Was it to stick a wrench into the gaming enjoyment of the 4e players that are just fine with the skill challenges and/or the errata presented?

For one last time: there's no hidden agenda, let alone a "cause" behind anything. As far as I'm concerned (I can't speak for others, obviously), Frank's post(s) argue that the amount of playtesting for skill challenges was unacceptable, from a customer's point of view, and I concur. Whence the need to pin an agenda to the making of this claim? It's straight forward customer feedback, the like of which you can read on any customer feedback forum, be it for electronic devices, car engines, or what have you.

As to the question why quoting that particular instance of customer feedback is salient in this thread, I can really just point you to the thread title. To be honest, it strikes me as seriously bizarre when I have to justify why I see fit to quote the material I did, and other people address all sorts of points (including my motives in doing so, over and over and over again) but not the one named in the thread title.

Is the point of this thread to state that a complex game has a subsystem that didn't quite work as well with 1 million players as it did with 500 playtesters? Don't all games suffer from the same problems?

Or was it to state without any proof that WOTC never tested skill challenges, neener, neener, neener, how-do-you-like-them-apples?

Are you seriously asking? After I've stated three times over what my intended, demonstrative claim is? And then offering this travesty of a question?

Regardless of context, it is ridiculous to claim that no playtesting occurred based on changes made after playtesting.

Where, in this thread, did I (or, for that matter, anyone else) claim this?

Defining the only playtesting you will accept as being playtesting of "the final version" is just a silly little dodge. It enables the speaker to almost always claim that no "real" playtesting occurred because playtesting almost always leads to changes and tweaks, therefore rendering the playtested version "not final."

Why do you think that? I'm curious. Personally, I would have thought that designers playtest material until they hit a version during the playtest - not after - when they say "yep, we're happy with that". In fact that's exactly my impression of a great amount of the combat system (and individualc lass powers) in 4E. The only point of contention from my side, actually, is to point out this discrepancy.

Just to mention a salient example. Know how some Daily powers in 4E have the keyword “reliable”? Well, thank the playtesters for that because WotC designers didn’t put it there before playtesters alerted to them that it’s hugely anti-climactic to have one’s daily’s (or at least some thereof) misfire. Guess what, WotC inserted the keyword, other playtesters confirmed that this was a good change, and so the final version kept it.* That, in a nutshell, is how playtest ought to work. Anything else ISN’T a playtest, it’s post-playtest adjustments to material.


*By the way, this is based on evidence of me having talked to a 4E playtester. I haven't been a playtester myself.

 

[Mallus]

4E combat doesn't appear to be excessively overpowered or underpowered, at least for most of the stuff in PHB1. I haven't got around yet to playing any classes or races from PHB2.

My group does uses material from most of the books. So far we've had to edit out some things, but by and large we find 4e balanced.

I do remember 1E AD&D magic users being very powerful at high levels. Though whether that was a deliberate design goal or a flaw, I don't really know.

Both.

For 3.5E, I never got the opportunity to create any munchkin overpowered characters. The DMs I played with banned almost all splatbooks and mainly allowed the core books only.

All you need is the 3.5e core books.

Most of the 3E/3.5E campaigns I played in, the DM ended the game at around level 8 or 9.

3.5e works well at those levels. Well, except the parts that don't, like multiclassed spellcasters, but on the whole sub-10th level 3.5e is a very good system.

However, as may be germane to the original topic, it's easy to point to parts of 3.5e and claim it was insufficiently playtested (for instance, by pointing out that multiclassing spellcasters --especially in a core-only game-- doesn't work).

 

[Mallus]

That, in a nutshell, is how playtest ought to work. Anything else ISN’T a playtest, it’s post-playtest adjustments to material.

What it is, is a bug fix. A patch.

This is the way product releases in the real world work. Early adopters are part of the beta test.

 

[ggroy]

3.5e works well at those levels. Well, except the parts that don't, like multiclassed spellcasters, but on the whole sub-10th level 3.5e is a very good system.

In one of my previous 3.5E games, the DM was receptive to continuing on after level 9, but with a caveat that he would ban any and all prestige classes. Unfortunately half of the players objected to this, and the game was brought to a final end. The DM thought that most prestige classes were either overpowered and/or outright silly.