4E playtesting or lack thereof

[catsclaw227]

For one last time: there's no hidden agenda, let alone a "cause" behind anything. As far as I'm concerned (I can't speak for others, obviously), Frank's post(s) argue that the amount of playtesting for skill challenges was unacceptable, from a customer's point of view, and I concur. Whence the need to pin an agenda to the making of this claim? It's straight forward customer feedback, the like of which you can read on any customer feedback forum, be it for electronic devices, car engines, or what have you.

As to the question why quoting that particular instance of customer feedback is salient in this thread, I can really just point you to the thread title. To be honest, it strikes me as seriously bizarre when I have to justify why I see fit to quote the material I did, and other people address all sorts of points (including my motives in doing so, over and over and over again) but not the one named in the thread title.

Well, apparently I am not the only one that thinks the Frank Trollman had an additional agenda, so it must not be odd of a question. He doesn't just argue the points. He wants to see mearls with crow in his teeth. He flat said this. So, it would be irresponsible to NOT question his motives because for 99% of the world, especially when it's a forum thread, a persons motives color their arguments.

Yes, the thread title asks if there was actual playtesting done on the 4e skill challenge system. But you do refer to his post, right at the beginning, as evidence that it wasn't. It is not bizarre in a debate to question the sources of a position. IANAL, but isn't that one of the things trial lawyers do?

I am also not the first person to question his statements as a basis of your argument. It must not be that bizarre to ask about it like I did.

For what it's worth, I like some of your takes on other 4e things, as read in your posts in other threads, but in my opinion, your claim isn't well supported for this one, so I question it.

Is the point of this thread to state that a complex game has a subsystem that didn't quite work as well with 1 million players as it did with 500 playtesters? Don't all games suffer from the same problems?

Or was it to state without any proof that WOTC never tested skill challenges, neener, neener, neener, how-do-you-like-them-apples?

Are you seriously asking? After I've stated three times over what my intended, demonstrative claim is? And then offering this travesty of a question?

I know what your stated claim is. The subtext in the argument is what I am questioning. I apologize for being snarky with the "neener,neener" part, but my first question was more what I was hoping you would state.

Isn't all this about the fact that, as I stated, a complex game has a subsystem that didn't quite work as well with 1 million players as it did with 500 playtesters? And that most all RPGs suffer from this issue?

What about the rest of my points? Do you have any comments on that, including how, in actual play, the skill challenges haven't been a problem?

Is the goal of your post simply to let us know how you feel about skill challenge playtesting? Are you looking for confirmation from WOTC? Is it to attempt to fix the problem and find a solution instead of having all of us grumble about it?

I know what you stated thesis is... what is the goal of the post?

 

[ExploderWizard]

What it is, is a bug fix. A patch.

This is the way product releases in the real world work. Early adopters are part of the beta test.

While this is true, it is an unacceptable way to test physical product unless replacements of "updated" content are given free of charge. Software can use that model much easier because the product is much simpler to patch.

It's like those bank commercials on TV. Even kids know that holding out on someone isn't very nice.

 

[AllisterH]

*Shrug*

If Blizzard can't produce a perfect product, no company can. Blizzard is well known in the industry as having the most extensive playtests around.

From their some time months long internal playtesting to BETAs that last several weeks, you would figure Blizzard would catch everything.

Hell no.

First month after release of the Lich King, the new death knight class gets some tweaks.

Some times playtesting will reveal a fault only after much time has passed. Take the monk for example. When the monk was first released, for months it seemed afterwards, did people call it overpowered.

Same thing with the Big Six items. When 3.0 was first released, the big six were not even a glimmer in the eyes of most players but these emerged after a couple of years

 

[catsclaw227]

That, in a nutshell, is how playtest ought to work. Anything else ISN’T a playtest, it’s post-playtest adjustments to material.

You are absolutely right about this. But like software, most game systems requires some patching to catch the things that a million users will see that 500 playtesters did not.

Errata is a real world adjustment. This really shouldn't be anathema to the players of RPGs, but we sure like to complain a lot about it.

Maybe because i am a software developer, I am softer on these kinds of issues...

 

[Obryn]

Well, I'll say that I think it's completely irrelevant why Windjammer posted the thread. I don't really care. I'm still looking for clarification on one or two comments, but whatever.

I am, however, still puzzled as to why playtesting is considered the end-all be-all of product testing; or why it's taken as a given that playtesting - even thorough playtesting - will catch all a game's flaws. I don't see a reason to believe that this is the case, and I can't recall an example of a complex game in which this was true.

Are there any?

-O

 

[ggroy]

I am, however, still puzzled as to why playtesting is considered the end-all be-all of product testing; or why it's taken as a given that playtesting - even thorough playtesting - will catch all a game's flaws. I don't see a reason to believe that this is the case, and I can't recall an example of a complex game in which this was true.

Are there any?

-O

Chess?

 

[Mallus]

While this is true, it is an unacceptable way to test physical product unless replacements of "updated" content are given free of charge.

But errata are free of charge.

As to 'enhancements' to the system which really address issues in the original design... well, they're going to cost you money. Note this is currently more true for the previous edition than 4e. A lot of later additions to 3e where meant to address various issues with the rule set, and for my money, much of that later material was good.

Software can use that model much easier because the product is much simpler to patch.

Now that's funny! Software patches may be easier to disseminate, they're not easier to write.

 

[Fifth Element]

Where, in this thread, did I (or, for that matter, anyone else) claim this?

From the OP, quoting you in another thread:

As he points out time and again, WotC has NOT playtested a single version of skill challenges to date (June 2008-June 2009) and has not released a version that is actually used by the designers in their home games, whether in a core book for which they charged $30 or an similarly pricey online service. Which is disheartening.

(emphasis added)

Technically, you did not claim this in this thread. It was in another thread, which was the inspiration for this thread.

 

[Mallus]

Chess?

Chess was an open-source project with a development cycle that lasted roughly 1500 years.

 

[ExploderWizard]

*Shrug*

If Blizzard can't produce a perfect product, no company can. Blizzard is well known in the industry as having the most extensive playtests around.

From their some time months long internal playtesting to BETAs that last several weeks, you would figure Blizzard would catch everything.

Hell no.

First month after release of the Lich King, the new death knight class gets some tweaks.

Software is a different animal. What if blizzard charged extra for the patches or using the patches required you run little sub programs along with the application ( stuffing paper errata sheets into a book)? Unless patches are seamless to the end user they are a burden and contribute to product failure.