WotC_PeterS talks about his "aggresive playtest" (with Le Rouse, SKR, & Noonan)

[Xanaqui]

Less playtest may be good.

I guess I've not done much game playtest (and even less was RPG playtest), but coming over from my professional field (real-time medical instrument engineering), large amounts of testing are the last resort of a company that doesn't know what it's doing.

A company that understands what it's doing should be able to catch the vast majority of defects before testing even starts - for example, from design inspections, implementation inspections, proofs, or from modeling.

Frankly, in any edition of (A)D&D, if I wanted to add a substantial novel new element or set of elements, I'd create a mathematical model, use that model to verify what the element did, and at most playtest to verify that there weren't large holes in my model (in other words, it wasn't the game element itself that I was testing; it was the model).

So I guess what I'm saying is that testing is only one means of verification, and it likely isn't a particuarly efficient one (it isn't in any engineering or mathematical field I've studied formally), so I'm not particuarly disappointed if there is less playtest in 4E than there was in 3E.

In fact, it's quite possible that that is a good sign.

Just to demonstrate that I'm not making this out of thin air; examples from General Principles of Software Validation:

Typically, testing alone cannot fully verify that software is complete and correct. In addition to testing, other verification techniques and a structured and documented development process should be combined to ensure a comprehensive validation approach.

Software quality assurance needs to focus on preventing the introduction of defects into the software development process and not on trying to "test quality into" the software code after it is written. Software testing is very limited in its ability to surface all latent defects in software code. For example, the complexity of most software prevents it from being exhaustively tested. Software testing is a necessary activity. However, in most cases software testing by itself is not sufficient to establish confidence that the software is fit for its intended use.

(Emphasis in original)

Replace the word "Software" with any other large piece of math (like, say, a complex game), and I think you'll understand my point.

 

[Mark]

There are no issues of mine. I'll repeat one significant quote: "We are not working on 4E"

I'm curious to see a link to a statement made by a WotC employee that contradicts what Scott has written, because whenever one has been called for in the past it either never appears, or the quote, when taken in context, does nothing of the sort.

http://www.google.com/search?hl=en&safe=off&q="We+are+not+working+on+4E"

 

[outsider]

Frankly, in any edition of (A)D&D, if I wanted to add a substantial novel new element or set of elements, I'd create a mathematical model, use that model to verify what the element did, and at most playtest to verify that there weren't large holes in my model (in other words, it wasn't the game element itself that I was testing; it was the model).

I'd be curious to see the mathematical model you would use to model "fun".

 

[Delta]

The stats were skewed; a 15 in AD&D meant nothing often times, but meant a +2 bonus in 3E. Those 15 scores then got shoved up to 16's often when you added the ability bonuses from every 4th level, so that's a +3. In the end, any 2E character I had, I ended up starting from scratch and converting by eyeballing it, the same method that James Wyatt recommends for 3E to 4E. It was an illusion of conversion that they realized wasn't necessary this time around. It's all in taking the concept, looking at the new rules, and then just figuring out what abilities give you the same feel as the character you had.

Well, I wasn't talking about conversions in my post. Nontheless, I used the written 2E->3E conversion for lots and lots of NPCs in adventures now in the ENWorld Conversion Library, with what I thought was very successful (and I got really good feedback on). Apparently that's a project that just has to get scrubbed re: 3E->4E.

But other than that, way back when I first saw the 3E ability scores, I said "Aha, clearly this is the most faithful linear interpolation of 1E ability scores". Scores of 15 in 1E gave a +1 bonus in most everything (Wis, Dex, Con, Cha), is +2 really so different? Doesn't the system resolve to the same +4 bonus for a score of 18 as in almost all 1E abilities? Is the +1/2 points not an exact conversion of late 1E/2E skill modifiers (for which each 1 skill point modified +2 to an ability check)? Have fighter attack bonuses not gone up +1/level identically all through 1E, 2E, 3E? Were combat details like charge bonuses and cover modifiers not exactly the same through 1E, 2E, 3E?

But far, far more important than that are the mass volume of text for spells and magic items (most of the PHB and DMG text respectively). Pick any 3 spells or magic items and I bet 2 of the 3 will be nearly duplicated text all the way through 1E, 2E, and 3E. It was with 3.5 that suddenly wholesale rewrites were being done to hundreds of the pieces of magic IP in the game. Now with 4E the entire system is itself being urban-renovated from scratch. So having those hundreds of pages of magic system wiped out makes my eyes go all Muppet-googly-eyed when someone claims that only minor systemic changes are being made.

 

[Rechan]

I'd be curious to see the mathematical model you would use to model "fun".

All I know is that it wasn't taught in my HS algebra class.

 

[Keefe the Thief]

Well, Scott Rouse´ little picture story made this thread archive-worthy. I wouldn´t have saved it for the rude questions and such, but that is just too cool to lose it.

 

[Xanaqui]

I'd be curious to see the mathematical model you would use to model "fun".

To me, the model itself would be "fun" However, note that I'm not a professional game designer.

I'd guess that a number of inter-compatible, easily learnable options that give different but comparable results under a specific set of preconditions would be a good place to start. Another way to put it would be to eliminate what isn't "fun" - forced algorithmic defects which mean that there is only one useful option would be an obvious example; forcing players into situations where they have no impact is another one. Obviously, that wouldn't take into account any "fluff"; that would require inspection and/or review.

I'm not suggesting that all testing should be eliminated; merely that optimally, testing the end product shouldn't be the primary means of verification or of removing defects.

Think of it this way: let's say you're playtesting 3.0, and you find a defect in Polymorph. It's a pretty big defect, affecting dozens of creatures, a bunch of spells, some class abilities, and some special ability descriptions. Well, you don't just loose the time that it takes to correct whatever issues you have with polymorph in the dozens of places it was spread throughout the game, you already have lost the development time, writing time, and some of the testing time on those items up to this point. If you had, instead, found the defect in early design, you would have lost, say, a few hours instead of hundreds. The closer you catch a defect to its point of injection, the more efficient your overall process is, and the fewer defects reach the product that you test - therefore, the less you need to rely on testing.

 

[Sitara]

:|

I think Scott shoulod change his JT from Brand Manager to Snark Manager.



(Look forward to Friday)

 

[portermj]

http://www.google.com/search?hl=en&safe=off&q="We+are+not+working+on+4E"

The sites I get from that link just have 3rd parties saying that someone from WOTC said they weren't working on 4E.

 

[Henrix]

Scott, I'm seriously worried about the guy you hired to manage DI, I think he might need a bigger gun!



(It's possible that he could get away with it if he were The Rouse, but there can only be one!)