It IS tricky to run good SCs and its fair to say that the SC section of the DMG was not 100% entirely baked out of the box. At least they did make some pretty significant errata. I'd also totally agree that the authors of the DMG (and to some extend DMG2 as well) seem to have issues coming to grips with certain key factors that make or break SCs. Perhaps they need to run more of them... Its hard to really consider that a "design flaw" in 4e as SCs are pretty generalized in format and good ones don't particularly have to step outside of the framework to be good. I think the fairest thing to say overall would be that 4e so far has not been all that adept at communicating the best way to run an SC. From the comments I've seen people make about them pretty consistently its readily apparent good SC execution techniques aren't widely known by DMs and a lot of what they do know seems to be wrong.
How long do you plan to make the "your DM sucks and can't do skill challenges right" excuse. It's been about two years now and the mechanical framework listed in the DMG and errata still does not work. The flaws are fundamental to the system and cannot be fixed. Here are the basic problems as I see them:
A. Skill challenges were billed as bringing the strategy of combat to non-combat situations. They don't. The mechanical strategies that skill challenges reward do not involve meaningfully working together or taking advantage of the situation described. All skill challenges reward the exact same strategy: Find the best guy to roll the required skills. Aid him if permitted to (and Mike Mearls podcast on skill challenges indicates that he at least thinks that you are permitted to aid other in skill challenges by the rules--so if those of us who think that is the case are deluded we're in good company). If you have to do something, try to talk your way into using your best skill. Never ever ever roll a skill that you are not good at.
That's it.
The analogous combat would be one where monsters had 17 different defenses and your job as a party was to guess a defense and attack it. Eventually you will win as long as you don't miss three times. It would be a lame system for combat. It is a lame system for non-combat challenges too.
B. Skill Challenges remove the players from the in-game situation.
Each character participating in a skill challenge participates individually. Consequently, the relevant question for a player in a skill challenge is not, "how do we work together overcome a concrete obstacle in the game world?" but rather, "how can I individually find a way to use a skill that I'm good at?"
There is no reward for coming up with a comprehensive plan that addresses the particular situation. (Ok, we'll go into a nearby shop and scout out the section of wall that we'll be climbing over--those of us who are good at it will haggle with the merchant over small items to give the rest of you an excuse to stand around and observe. Then, when we've figured out the guard schedule, you distract the guard a block down the street and we'll climb over the wall while they are busy. Then you wait five minutes and teleport to the top when the guards have gone past. We'll catch you when you jump. That way, we won't have to worry about how Sir Stoneshoes will sneak past anyone in his armor). Instead, the rogue who is good at stealth says, "OK, I'm super stealthy so I sneak up to the wall." He rolls a stealth check. One success. If he's clever, Sir Stoneshoes may pretend to be in on the story and say, "I'll climb over the wall" and roll an athletics check quickly before anyone asks him how he sneaked up to it. The wizard isn't any good at sneaking or climbing but he's an illusionist and has good bluff thanks to multiclassing, so he says, "I'll distract the guards" and rolls a bluff check. Only one more success needed, thinks the cleric: "I'll roll a perception check to figure out the patrol schedule." Oops. He rolls a 1 and fails. Apparently he miscalculates the schedule so someone is caught climbing over the wall (it must be Sir Stoneshoes because he's the only one who actually tried climbing). But it's only one failure so the challenge isn't over yet. Ever resourceful, the rogue thinks, "I'll light the inn across the street on fire so the guards have to run to put it out and let us go." So, he makes a thievery check and the party beats the skill challenge. But the only reason that it works at all is because the scenario was not really imagined, but rather was glossed over. If the scenario had been fully imagined, all failures would not be equal (failing to climb the wall might result in falling, but getting caught while sneaking up to the wall would require some quick thinking if it wasn't going to result in the failure of the entire enterprise). Likewise, if the scenario had been fully imagined, some of the rolls would have been unnecessary (the rogue wouldn't need to sneak if there was no-one there to see him and the wizard wouldn't need to distract the guards if they weren't there either). Finally, if the scenario had been fully imagined, the players would have not been able to avoid rolling some skills they would rather not make. The wizard either needs to teleport, fly, or climb up the wall--he wouldn't get over it just because the fighter rolled an athletics check.
C. The presumed scaling for DCs either leads to ridiculous results (the super-athletic high level fighter still fails to climb a simple brick wall 30-50% of the time) or prevents the use of certain types of role-playing scenes (it would be an interesting role-playing challenge to see how the high level fighter with minimal social skills tries to talk his way past some indifferent guards, but that does not fit in the concept of a skill challenge). The former situation occurs if you try to use scaling DCs for things that the designers would probably argue no longer constitute a proper challenge for high level characters. The latter situation occurs when you have decided that things that should be simple for
competent high level characters are no longer appropriate material for skill challenges. Outside of the skill challenge format, the fighter would pit his low numbers against the low fixed DCs in order to resolve the latter situation, but once you have established the ground rules that say certain challenges are not worthy of a skill challenge, that situation doesn't qualify. Skill challenges have to be keyed to characters who are good at the relevant skills in order to work. So you they don't work to describe situations that are only challenging because they address a character's weaknesses.
D. The default model of the skill challenge is "Failure does not get the PCs off the railroad." In other words, players should not fail to move to the next scene of the adventure for failing a skill challenge. Rather, they should suffer some kind of penalty (surprise round for the enemies, lost healing surges, etc) and move on. This aspect of core rules skill challenges minimizes the significance of the non-combat aspects of the story. The players can't ignore all the clues and never find the malefactors of the story. All they need to do is fail enough times and they'll get where they wanted to go.
E. Therefore, the primary effect of skill challenges
as described in the DMG is to gloss over the non-combat exploits of the PCs. While they were sold and advertised as a means of mechanically handling non-combat situations, what they actually accomplish is to provide a way to quickly get non-combat situations out of the way by rolling a few dice and glossing over all the messy details.
If skill challenges were to be successful, they would need to abandon the one-size fits all situations mechanics found in the DMG and errata and create a custom mechanic to model each non-combat situation as it arose. They would need to ask the PCs to work together to overcome concrete obstacles where different tasks required different resolutions and entail different consequences for failure. The successful mechanical resolution of non-combat situations that I have observed in 4e has done that. But despite usually being called a skill challenge, they have (correctly) discarded nearly every part of the DMG skill challenge mechanic--most especially the "name any skill you want", the "three strikes and you're out" aspect, the "all DCs scale to player level on a 5/10/15 scale" aspect, the "all skill checks entail similar consequences for failure" aspect, and the "failure still moves you to the next scene in the story" aspect.
Skill challenges were an admirable attempt to come up with a unified and interesting system for resolving non-combat encounters. It failed completely and utterly and on every level.
