D&D 4E Heavy Concrete Data on 4e's Skill Challenge System (long, lots of tables)

Dave Turner said:

In other words, if we assume a conscientious DM who tailors a skill challenge to his party using the "unfootnoted" DCs, the problem seems to be substantially reduced. If it's reasonable to assume a conscientious DM, then doesn't Ockham's Razor point us to an innocent error by WotC with the +5 DC footnote?

Click to expand...

I think that Table 2 shows that if you use the +5 DC in the footnote and assume that each PC will have bad assistance from the other PCs (PWRBA) then the problem seems to be substantially reduced leaving success in the region of 80% to 100%. If only three PCs assist you go to 50 to 100%.

I know I don't post often on this forum and usually stick to the background - especially intensely mathematical threads like this. However... I have read over and over again that creating a game system is not just math - it's also an art. Is it possible that just because the math says one thing, that doesn't mean the system itself is inherently flawed? They've obviously tested the skill challenge system a lot - WotC talked about it multiple times before releasing 4e. They had playtesters.

It is actually very, very easy to create a mathematically perfect system. You can alter DC's, account for certain variables, etc. and (mathematically) scale things exactly how you want them on paper. But that does not mean that that's how it always plays out. I would be incredibly surprised if WotC didn't make a "perfect" system at some point and decided to go with this instead.

Basically my point is just this: a role playing system isn't just about math, it's also an art. The dice actually don't roll like the paper says they will. Actual success is not necessarily guaranteed, even when the numbers say 100%.

The other night my group and I ran a practice 4e combat session. I (the DM) got two critical hits early on. I rolled consistently higher than 12 all night. I only missed the players a grand total of three times (I think that was it). On the other hand, they rolled multiple 1's, and had difficulty rolling above a 3-5 for some reason for more than 3/4 of the night. They almost died in a battle that should have been incredibly easy for them to overcome. Why? Horrid, horrid rolls.

The situation has been reversed many times, too.

My point: I don't really care what the math says on paper. I care about how things actually play out in game, and the math is absolutely, completely no guarantee about how things will actually play out in the game. That is an art, not a science. I'm going to stick with the skill system as is, and I would recommend others do the same. If, throughout the course of the game, we discover that it is horribly broken (one way or the other) - then maybe we'll try to do something about it. But I'm getting kind of tired of these threads talking about how 4e is broken when it hasn't even been out for a week, yet, and people haven't seen it in play for any period of time at all.

Come on... can't we give WotC the benefit of the doubt until we see (in game, as the pieces are fitting together) otherwise? Innocent until proven guilty?

That's what I plan on doing, anyway.

Rystil Arden said:

It's still not clear to me that Aid Another is supposed to be usable in a Skill Challenge. At least, if it is allowable, then they've put in some redundantly pointless options, since as you said, Aid Another is quickly an auto-success (and the options they include for + or - 2 are Easy Checks, which can usually still fail at least some of the time).

Click to expand...

Exactly. I don't think it is allowed. It goes against the stated goal of having multiple players participate. If we're just seeing how high the main skill guy can roll, why are we even engaged in an initiative situation. I don't think most people are parsing the words carefully. It's as clear as it can get without specifically stating that Aid Another isn't allowed.

DMG p74 said:

In a skill challenge encounter, every player character must make skill checks to contribute to the success or failure of the encounter. Characters must make a check on their turn using one of the identified primary skills (usually with a moderate DC) or they must use a different skill, if they can come up with a way to use it to contribute to the challenge (with a hard DC).

Click to expand...

DMG p76 said:

Skill challenges require the players to make rolls at specific times. Call for these checks according to the pace of the narrative and the nature of the challenge. This might be each round on their turns, during each short or extended rest, or some other time frame as determined by the challenge in question. Skill checks usually count as successes or failures for the challenge, but sometimes a specific use of a certain skill in a challenge just provides a minor benefit or penalty.

Click to expand...

This says to players are required to make skill checks. Aid Another is a standard action on p.287 in the PHB that allows you to aid another character's skill check. Aid another is not a skill check, it's another type of action. Depending on how the DM is running the encounter, he may allow you to Aid Another in ADDITION to the required skill check, but I would think that would be mentioned if it were an option, and it would require the other actions in the challenge to be considered the equivalent of a move action.

Not only that, but even if Aid Another were considered to be a skill check, then it would automatically also have consequences for failure. True, it's dirt easy with a DC 10, but I don't think that's being accounted for.

The only skills that don't have consequences for failure are the ones specifically listed in the skill challenge, and those provide a -2 on a failed roll (and not with a DC 10, mind you). Why would that option be present if Aid Another were allowed, when it's automatically superior?

It seems clear to me that if you're involved in a skill challenge, you have no choice but to roll and risk failure. That's not to say that everyone in the party must be involved in the skill challenge. If the Ranger is running a good 50 feet ahead of the party tracking, he's the only one making checks. If other people are making checks too, they run the risk of stomping over important clues, and ruining things, unless the DM wants there to be a specific option for them to aid him.If you talk at the Duke's dinner, you run the risk of ticking him off - simple. If you're running the Urban Chase challenge, you're either chasing or you're not.

Finally, it's obvious that Aid Another has a significant effect on the numbers if allowed, with its easy DC 10. If it's allowed in a skill challenge, why isn't there even one mention of it as a valid tactic in the text? It should be noted that, in the example, Kathra's +2 granted to an ally seems to be a bonus granted by a DM who liked her response, as her success is still tallied as one of the group's successes, thus it isn't an Aid Another action. Also, it would have had to be a Bluff check, rather than a History check to work as an Aid Another.

I think it's clear that you can't Aid Another to skip out on your skill check. If a skill is specifically listed as having not counting as a failure and granting bonuses, such as Perception in the Urban Chase challenge, that's the only option.

Skywalker said:

I think that Table 2 shows that if you use the +5 DC in the footnote and assume that each PC will have bad assistance from the other PCs (PWRBA) then the problem seems to be substantially reduced leaving success in the region of 80% to 100%. If only three PCs assist you go to 50 to 100%.

Click to expand...

Right, I think we're in agreement about Table 2. It does include the +5 DC footnote. But the numbers in PWRBA average out to around 95% or so, after we account for the wonkiness of 1st and 7th level (WTF is up with 7th level in Table 2 anyway??). That's not quite the same as an 80%-100% range. After level 1, we're basically talking about success in almost all cases, according to PWRBA.

In Table 4 (which Stalker0 generously provided), we can see that there seems to be a significant breakpoint between 2 PCs Aiding and 3 PCs Aiding. The contrast is mostly pretty stark. The system (assuming the +5 DC boost) virtually requires that PCs Aid each other if they're to have any reasonable chance of success. Why doesn't the DMG or PHB warn anyone about this "feature" of the skill challenge system? Either way it's an oversight by the developers. Either they failed to properly analyze their math and include this crucial tactical factoid about using Aid Another or they made a mistake by making the inclusion of the +5 DC far too ambiguous.

Dave Turner said:

In other words, if we assume a conscientious DM who tailors a skill challenge to his party using the "unfootnoted" DCs, the problem seems to be substantially reduced. If it's reasonable to assume a conscientious DM, then doesn't Ockham's Razor point us to an innocent error by WotC with the +5 DC footnote?
EDIT: I just remembered, Stalker0, that your assumptions are for a Complexity 5 challenge (12 successes v. 6 failures). Do the probabilities worsen, for ALL tables, if the complexity starts to drop?

Click to expand...

If all you are looking for is to get the win rate up into a ball park 50-80% bracket, then yes taking away the +5 DC will greatly help the skill system, though I would still bump up the epic level DCs, they are too easy. So if WOTC came and said that the +5 DC was an error, I could believe it.

However, there remains a secondary problem, one of variation. If your party deviates from that perfect medium by even a little bit, it can have a tremendous impact on your PWR. The effects of high skill individuals, and how much allowance your dm has for aid another has a very large impact on the PWR. This is a much harder problem to fix.

As to your question about complexities, as long as the IC is less than 70%, then lowering complexity will increase PWR. So from those tables, the low PWRs will get bumped up a bit, and the ones that are close to 80% will rocket off to 100% that much quicker. IF your IC is 70% or higher (such as in Table 3), then lowering complexity will decrease PWR.

Eldorian said:

But it's not, really. What Stalker0 is doing isn't hard. I taught a course in exactly how to do this stuff last semester, aimed at university freshmen and sophmores.

Click to expand...

I agree the maths isn't hard. However, the problem is that most RPG system will produce a string of results based on the amazing amount of variables they contain. 99% of play tends to happen in the a middle band of 50% of those result. The 25% either side can be very difficult to control through RPG design. I am sure if you ran models of combat which is far more complex you would get quite results at time.

So Stalker0's analysis is excellent and very informative but really only actual play will provide a true test of the system. Having seen some of the middle ground, I am becoming more confident that for the most part the Skill Challenges will run fine with a conscientious GM.

Dave Turner said:

Either they failed to properly analyze their math and include this crucial tactical factoid about using Aid Another or they made a mistake by making the inclusion of the +5 DC far too ambiguous.

Click to expand...

Or they knew exactly what they were doing, the system works great as is, and all of this math only works on paper and not in game.

Logically speaking, that is a third option.

Stalker0, would it be possible for you to run a table based on one of the existing official challenges?

I'm curious to see if a) they followed their own rules and b) if it matches your expectations of a standard challenge.

One of the things I think has been missing.. BADLY missing, is the examples from the 3.x books. They were well done, and introduced a lot of feel to the rules.

I like what I see...

First, thank you Stalker0 for the all the work you put into the tables.

I gotta say, I like the numbers I'm seeing.
1. The systems tendency to vary wildly if the party deviates from the norm makes the system interesting to me as a DM.

2. Skilly McAwsome is clearly a necessary party member if the party wants to win skill challenges about as often as they win combats. IIRC, the DMG compares skill challenges to combat and makes references to that effect. So, if the party is winning almost 100% of the time if they all work together then that makes perfect sense to me.

With a PWR at about a 100% with good aid another (assuming 4 aids), I'm thinking of using skill challenges as an aid to banter with NPCs.

Stalker0 said:

If all you are looking for is to get the win rate up into a ball park 50-80% bracket, then yes taking away the +5 DC will greatly help the skill system, though I would still bump up the epic level DCs, they are too easy. So if WOTC came and said that the +5 DC was an error, I could believe it.

However, there remains a secondary problem, one of variation. If your party deviates from that perfect medium by even a little bit, it can have a tremendous impact on your PWR. The effects of high skill individuals, and how much allowance your dm has for aid another has a very large impact on the PWR. This is a much harder problem to fix.

As to your question about complexities, as long as the IC is less than 70%, then lowering complexity will increase PWR. So from those tables, the low PWRs will get bumped up a bit, and the ones that are close to 80% will rocket off to 100% that much quicker. IF your IC is 70% or higher (such as in Table 3), then lowering complexity will decrease PWR.

Click to expand...

Ok, then I can safely say that I (a non-math person) understand what's contained in the OP.

I still find the implications of your last sentence fascinating. More competent characters will succeed less as the complexity goes down (if I'm reading that correctly?). Hardly intuitive!

You're a gentleman and a scholar, Stalker0.