Skill challenges are nothing new to D&D. I was using them all through 3.5. It is just being touted now as a new shiny for the newest incarnation.
Well, more specifically, the new rules tend to encourage it and present it as an option, whereas it was entirely on the DM to come up with the concept in 3.5, and the rules themselves somewhat worked against it.
For myself, I find skill challenges to be a brilliant concept with a massive amount of potential... that require a lot out of the DM to pull off well.
Now, there really are two types of skill challenges. One of them (such as the one in the article) is the planned scene, and often an extensive one - these tend to require mainly preparation. If properly designed, however, they then flow smoothly and easily, allowing PCs a vast array of options and a very dynamic and engaging encounter, without too much actual challenge
running them for the DM. But, again, it requires doing a good job designing the options from the start, and avoiding making it too linear or restrictive.
The second type is the more improvised challenge - often showing up when the PCs try to do so something slightly outside the box, but also an option for a DM who wants an interesting encounter, but doesn't want any limits at all on how the PCs will solve it. Instead, the DM simply decides what the scene is, what the goal is, and how difficult it should be, and then lets the PCs solve it in whatever way they want to. This is equally challenging for the DM, but the requirement of the DM here is improvisation - can they take the disparate actions the PCs are doing and find a smooth way to meld them together? Can then have the encounter dynamically changing on the fly to adjust to the PCs actions?
If successful, this can be an even more rewarding experience, since the PCs really feel like they are in the heat of the moment, with their actions and personal choices having immediate and direct results. But, again - this really needs a DM who can make decisions on the fly, or else you end up with a muddled and confusing scene, with half the PCs discovering their actions are useless, and no one entirely sure what they are even supposed to be doing.
I'm not sure more playtesting would really help - the core rules for them are very solid, save for one small footnote on the table. (And the errata they delivered due to that footnote ended up going way overkill.) The issue is simply that they can take a good deal more difficulty to run well than a simple combat does, and that there really isn't enough guidance on putting them together for new DMs.
I think the current articles by Mearls will help with that (for those with DDI), though I do hope that they will also provide some focus for the improsived challenges, along with the advice they are currently giving for planned challenges. I certainly don't think the planned challenge in any way defeats the original concept, though.
On a final note, there is one other big hurdle to understanding how to run a skill challenge - the concept of failure. PCs are expected to win 99% of combats, in general. This obviously changes from one campaign to another, but in general, it is a rarity when a party gets wiped out - or even captured or forced to retreat.
Skill challenges, on the other hand, are designed to have a much more tangible failure rate. PCs should still succeed more often than note, and probably even most of the time - but failure will occasionally come. And this means you need to both run it in a way that failure doesn't make the players bitter, and also carries consequences that they find acceptable - either simply the loss of some healing surges / treasure / etc, or possibly a new dangerous situation they must handle with a combat (or even another, different, skill challenge!) Some challenges might not even have a true penalty for losing - just grant a bonus for success. Understanding how to weigh the success and failure of a challenge is also something I think many overlook, and can give a poor first impression of the system when it does, in fact, go wrong.
