Would you be willing to restate his argument in your own words, since you're advancing it?
With pleasure!
Massawyrm’s very favourable review of his playtest of 4th edition included a glowing review of skill challenges.
I don’t know at what date he carried out his playtest (and whilst the impression I get is that he wrote about it shortly afterwards, I don’t have a date for his post either). However, at some point after that but before sending the DMG to the printers WotC changed skill challenges.
The rules in the DMG no longer include rolling for NPCs “counter-arguments” and the range of possible results (which Massawyrm seemed to particularly like) was replaced by a simple pass or fail.
Neither Massawyrm nor anybody else, to my knowledge, has posted about playtesting the DMG version. This may be due to the terms of NDAs. Mearls could have clarified the situation if he wanted to, but chose not to. This is all “absence of evidence is not evidence of absence” territory but at least raises the possibility that they didn’t playtest the DMG version.
Clearly, the DMG version was not playtested sufficiently, as it required correcting soon afterwards. I am really not interested in the mathematical side of D&D, but apparently skill challenges were either far too hard or far too easy, depending on the circumstances.
I am very hazy on the details, but the consensus on EnWorld is very much that they needed fixing, and indeed they were corrected by WotC very quickly – apparently in response to the concerns of players.
It is Frank Trollman’s contention that even a cursory playtest would have identified the problems in the DMG version, as they are glaringly obvious. They aren’t that obvious to me, but then I haven’t played through a skill challenge.
Thus either WotC didn’t playtest the skill challenges which appeared in the DMG, or they chose to publish something they knew was wrong, or the playtesters failed to spot something that a great many other people (apparently) spotted straight away.
