D&D 4EShowing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)

Stalker0

Legend
Showing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)

Showing the Math: Proving that 4e’s Skill Challenge system is broken.

I have created a new skill challenge system to try and correct the problems with the existing one. I was kindly asked to prove that the original system is flawed, and I am happy to do so.

This thread is very math heavy. I assume you are proficient with basic probability and have the ability to understand combinations.

The Scenario: We will use a party of 1st level adventurers, and give them a Complexity 1 (Success 4 vs Failure 2), Level 1 task. Basically the easiest skill challenge such a party could face. I will assume a skill challenge where all of the DCs are medium, as this is the standard default for challenges. Then I will assume every player gets to use one of his best skills each time. For most characters, that’s probably +9 (+4 from a stat, +5 for skill training). However, that gives me a 50% success rate, and a 50% failure rate. That can get confusing in math equations, so I will give the player’s another +1 advantage, making it 55% success and 45% failure. The question is, how likely is a party to succeed at this skill challenge?

The Math: Conditional vs. Non-conditional Probability
There are two main ways to calculate the odds of this scenario. One is conditional. That means I take into account that fact that once I hit two failures, the skill challenge is over. That could happen in as little as 2 rolls, it could happen in as many as 5 (if I’m rolling a 6th roll, I would have already had either 2 failures or 4 successes, so 6 rolls can NEVER happen).
The other way is non-conditional. What I can say is I’m not going to look at when the failures happen, I am simply going to look at ALL the scenarios in which the party would beat the skill challenge, and discard scenarios where they would fail. This method is probably less intuitive right now, but I find the math easier to do, so I will choose this method. I will walk you through it as I go, and it should make sense.

Calculating the Odds:
What we are going to do is look at ALL the scenarios in which the party could win this skill challenge. The first thing to note is that the maximum number of rolls that can occur in this scenario is 5. At 5 rolls, I have already determined if the scenario is a failure or a success, every time. For example, if I had 3 successes I must have had 2 failures and the challenge has failed. If I have 4 successes and 1 failure, then the challenge was won.

What I will assume is that every skill challenge is a series of 5 rolls, and I will add up ALL of the scenarios in which the party gains a success. The reason I can do this: Because I’m looking at only the winning scenarios, the maximum number of failures I can have is 1. After all, if I have 2 failures, I would have had to fail the challenge somewhere along the way through the 5 rolls. Therefore, I don’t care when the failure happens. I could happen in the 1st roll, or the 5th. As long as I have my 4 successes in their somewhere, I will count it as a win for the party.
So in our series of 5 rolls, there are two ways in which the party can win the skill challenge:
1) 4 successes and 1 failure.
2) 5 successes and 0 failures.
We must calculate the odds of both and add them both together to get the total party success.

4 Successes and 1 Failure
First, we calculate the odds of this happening once. From our scenario above, the party has a +10 to their rolls, and they are rolling against DC 20 (which is given in the DMG pg 42). That means they have a 55% chance to make that DC 20 every time. That means they will also have a 45% failure each time.
So the probability of getting 4 successes and 1 failure is: .55 * .55 * .55 * .55 * .45 = .041.

Now we have a single probability, but we have to account for all the scenarios in which 4 successes and 1 failure can occur. For example, you could have SSSSF or FSSSS or SFSSS or SSFSS or SSSFS. We will use a combination to calculate this, because the position of the successes and failures don’t matter.
That’s C(5 rolls, 4 desired) = 5. So we will multiply our initial probably .041 * 5 = .205.

5 Successes and 0 Failures
Initial Probability: Now its .55*.55*.55*.55*.55 = .55^5 = .0503.
Our combination is C(5rolls, 5 desired) = 1. 1 * .0503 = .0503
Total Outcome: Adding both of these scenarios together gives us: .2508 = 25.08% win rate. So basically with my party taking on the easiest possible challenge, they have only a 1 in 4 chance of succeeding. And remember, I gave them an extra bonus!! If I use my original example (+9 to each roll) they only have an 18.75% chance.

Calculating Higher Complexities
So I have shown the math for the most basic challenge. But often you are going to want complexity 5 challenges, which is supposed to be the skill challenge equivalent of a battle. A battle against basic level 1 guys doesn’t sound too bad right?
I’m not going to go into the full math again, but I will lay out the ground work so you can do it yourself.
We need 12 successes before 6 failures. That means at most we can have 17 rolls (Notice how this number is always one less than the total between successes and failures, that never changes). Out of all of our 17 roll scenarios, which ones will give us success?
The answer, 12 successes and 5 failures, 13 successes and 4 failures, 14 successes and 3 failures, 15 successes and 2 failures, 16 successes and 1 failure, and 17 successes and 0 failures. Phew!! A lot of math to do. I will run the first scenario, and leave it to you all to finish the rest. I will of course provide the final answer!

12 successes and 5 failures.
Our individual probability is .55^12 * .45^5 = 1.41E-05 = .000014
Our total probability for this scenario is C(17 rolls, 12 desired) = 6188. 6188 * .000014 = .08749.
So now would repeat the process for the other scenarios. Doing that our total probability is: 14.71%, which is about 1 in 7. Again, that’s using that extra bonus I added, what would it be if it was a straight up 50/50 roll each time? 7.17%, about 1 in 14.
As a DM, do you feel that giving your party a 1 in 14 shot of handling even a basic challenge is fair?

Taking it to the Next Level. Well we’ve seen that a 1st level party can’t handle its own challenges, so let’s bring in our boys from 4th level to do the job right. Those guys have a massive +2 bonus to all skills to help them out, so a total of +11 (I don’t think these guys need that freebie bonus, afterall, these challenge is literally beneath them in difficulty!)
4th level party vs 1st level, Complexity 1 Challenge: 33.70%
4th level party vs 1st level, Complexity 5 Challenge: 26.39%
Um…maybe that party needs that freebie bonus afterall?
How about 6th level for another +1?
6th level party vs 1st level, Complexity 1 Challenge: 42.84%
6th level party vs 1st level, Complexity 5 Challenge: 41.97%
Okay okay, enough of this! Our 8th level guys come in with another +1 and their stats have been bumped again, so let’s give them a net +2 over our 6th level team.
8th level party vs 1st level, Complexity 1 Challenge: 63.28%
8th level party vs 1st level, Complexity 5 Challenge: 76.53%
Your going to look at the 8th level numbers and think it’s a typo…its not. That’s a result of how the complexity numbers are laid out. Once you get to 70% chance of succeeding on each individual roll, higher complexities actually give you a higher chance to succeed.
So…we have finally gotten a better than 50/50 chance to beat the challenge, and it only took a party 5-7 levels higher to do it.

Conclusion
I hope these examples have given some concrete proof that skill challenges are flawed as written. If you have any skepticism left, remember that I gave the party the advantage; I assumed they could use their best skills for the job. Take my original example, assume a 5 person party, and let’s say that one character only has a +7 to his skills for that challenge. The result against a complexity 1 challenge? 16.35%. Against Complexity 5? 5.17%. Better roll a 20 on that d20, because that’s all the shot you have.
However, there is hope. I have taken the time to reconstruct the model and fix the numbers. That system is in the thread above. I hope this article has been a good eye opener for everyone, and I hope that if you don’t choose to use my system you do find a new system that works for you. While the math of the skill challenge system doesn’t work, the concept is very good, and I would hate if any group gave up on it due to numbers alone.

keterys

First Post
As far as I can tell, the 'Add +5 to the DC' for skill challenges is in error. The math arights if you remove it.

Ie, 10/15/20, not 15/20/25.

10/15/20 is also what was used at D&D XP.

Surgoshan

First Post
At first level, the easy/medium/hard DCs are 15/20/25.

So your PC with +9 on his skill rolls will need 6/11/16 to succeed at them. Given that they can only get one failure, it just makes sense to focus on easy rolls. For the more complex challenges, with greater leeway, they can go for harder rolls.

So to get four successes in a row (NOT five) is .75^4 = 31.6%. All of a sudden, skill challenges don't look so impossible.

keterys

First Post
... as far as I know, you can't just choose to do easy challenges. I might be wrong. Basically you unlock them or certain specific skills are easy, but every member of the party is almost assuredly not doing an easy challenge.

Stalker0

Legend
keterys said:
As far as I can tell, the 'Add +5 to the DC' for skill challenges is in error. The math arights if you remove it.

Ie, 10/15/20, not 15/20/25.

10/15/20 is also what was used at D&D XP.

Taking off the +5 definitely helps, though that's not the end of the story.

Basically, once you get into range with your DCs, the complexity numbers flip. At Complexity 5, the party would have a 76% chance to succeed, and complexity 1, a 63%

And most skill challenges assume medium rolls, easy rolls are not something you can just use.

hong

WotC's bitch
You forgot take 10.

Storminator

First Post
Your model of success is slightly flawed. There are two possibilities for success in the complexity 1 challenge:

1) 4 successes and 1 failure
2) 4 successes and 0 failures

You have #2 requiring an additional success. This doesn't change the answer significantly. You also haven't allowed any easy checks, nor accounted for aiding another (does that rule still exist?).

I do think you've got the fundamentals right tho. It looks difficult.

PS

Ydars

Explorer
Stalker; your math doesn't seem to include the fact that some players can roll a skill that does not contribute to successes or failures (called a secondary skill in the DMG) but does give other party members +2 to other related skill rolls (see DMG)

Does it also allow for the fact that up to 4 players can co-operate to give the person rolling an up to +8 circumstance bonus if they make a DC check with their skills (p179 PHB)?

Obviously you can't do this with every check, but inventive PCs will soon think of ways get these and extra +2 circumstance bonuses to make these skill challenges possible and even highly probable.

It is also mentioned in the DMG that certain uses of rituals might grant automatic successes etc.

Stalker0

Legend
Storminator said:
Your model of success is slightly flawed. There are two possibilities for success in the complexity 1 challenge:

1) 4 successes and 1 failure
2) 4 successes and 0 failures

You have #2 requiring an additional success. This doesn't change the answer significantly. You also haven't allowed any easy checks, nor accounted for aiding another (does that rule still exist?).

I do think you've got the fundamentals right tho. It looks difficult.

PS

You have to remember that I am looking at a non-conditional model. In your example, 4 successes and 0 failures mean that the players roll 4 die and then stop immediately, just as if they roll 2 failures out of the box they would stop. In my model, you are always rolling 5 rolls, and seeing how many of them are successes.

Both conditional and non-conditional models work for this...but the math is VERY different. You cannot place ANY conditional modifiers into a non-conditional equation. That is why I have 5 successes and 0 failures.

I didn't take into account easy checks because the default skill challenges don't use them, they are all medium.

Aid Another does exist in a limited form. If its supposed to be the standard in skill challenges, its not explained correctly. If for example, a group of 5 players can have 1 person make rolls, and the other four provide +8 bonuses (which is the example given), then the skill challenge is so easy you might as well not even bother.

Mistwell

I thought the use of secondary skills and aid-another was explained well in the DMG, and expected. I don't see how you can hand-waive them if you are going to crunch the numbers.

Stalker0

Legend
Mistwell said:
I thought the use of secondary skills and aid-another was explained well in the DMG, and expected. I don't see how you can hand-waive them if you are going to crunch the numbers.

They explain aid another in one section, but don't go into with their skill challenges sections. Even in their example they don't use aid another, so I'm not sure when you are supposed to add it in.

Further, aid another becomes extremely easy in short order. For example, by level 4 most characters can autopass aid another rolls in their good skills, heck some character can do it at level 1. In such a case, the lead guys gets a +8 bonus, which then makes the skill challenge extremely easy, and boring as the rest of players just aid. If that's how WOTC intending skill challenges to be done then my numbers would need to be fixed, and I would say that skill challenges are too easy

As for secondary skills, many challenges don't have them, or have a combination of hard and easy skills. But I think your right that it would be good to see how those easy DCs affect things, I will run some more numbers and get back to you with them. Probably won't be today though

Storminator

First Post
Stalker0 said:
You have to remember that I am looking at a non-conditional model. In your example, 4 successes and 0 failures mean that the players roll 4 die and then stop immediately, just as if they roll 2 failures out of the box they would stop. In my model, you are always rolling 5 rolls, and seeing how many of them are successes.

Both conditional and non-conditional models work for this...but the math is VERY different. You cannot place ANY conditional modifiers into a non-conditional equation. That is why I have 5 successes and 0 failures.

I forgot you also had SSSSF, which, when coupled with SSSSS, covers all possibilities of the 5th roll.

I didn't take into account easy checks because the default skill challenges don't use them, they are all medium.

That's not exactly what it says. It says to use moderate as a starting point, but point taken.

Aid Another does exist in a limited form. If its supposed to be the standard in skill challenges, its not explained correctly. If for example, a group of 5 players can have 1 person make rolls, and the other four provide +8 bonuses (which is the example given), then the skill challenge is so easy you might as well not even bother.

In addition to the group check, the multiple players section talks about cooperating to help other players make checks. Sounds like aid other to me.

The samples also include easy checks that don't do anything but add a modifier to other checks. There's also advice about giving out +2 for good ideas.

:shrug:

PS

Tervin

First Post
Your post made me look closer at skill challenges, and regrettably you are right. As written they are a great idea that falls apart because of bad math. One simple house rule would be to allow taking 10 even though that is normally not allowed inside an encounter. That would take away some of the fun of the whole thing though.

I like that you added bells and whistels to your system, but I don't think I will want those added features from the start. They can be something to add for players who like skill challenges and want to have more things to do in them.

The rules as written have two big weaknesses:
1. Skill challenges are unnecessarily hard, punishing suboptimal party builds even more than battles do.
2. The system for complexity is useless, as more complex challenges are easier than less complex ones if difficulties are set as easy, while more complex ones are harder than the less complex ones if the difficulties are set really high. Giving more XP for more complex challenges makes sense as they involve more roleplaying, on the other hand it has nothing to do with difficulty.

I will have a closer look at your alternative system, but before going there I want to try a simple method: Being generous with RP bonus to skill rolls during challenges. If the players are reasonably imaginative in how they RP their skill uses I will give them +2-4 to rolls, which should make it possible to actually succeed (as long as they use "aid another" cleverly). This way they also have to work harder in a complex skill challenge, and thus deserve their XP.

Kaodi

Hero
When I run the numbers, I get ~0.297 for Complexity 1 with 55% Success, and ~0.195 for Complexity 5.

First Post
Does this math match up with anyone's play experiences? I have trouble believing that playtesters worked at this game for as long as they did without ever noticing that their party fails 80% of all skill challenges. Perhaps some factor is being omitted?

nittanytbone

First Post
It seems to me that for skill challenge design, the DM needs to be sure to include some Easy tasks, or tasks that provide bonuses to others.

For example, a successful Perception check (Easy) might grant a +2 non-stacking bonus to the all future Thievery checks until the end of the challenge.

Additionally, I think the designers deliberately low-balled things and made the skill challenge system HARD under normal circumstances. This allows the DM to hand out +2 bonuses for good ideas like candy and accounts for the players picking up magic items with significant skill bonuses without anything breaking.

+9 is perhaps a "most likely," but +5 trained +4 attribute +2 racial + 3 skill focus is not out of the question, for a total +14 mod. Throwing in a +2 circumstance modifier and a magic item jacks it up even higher. Now that players realize that skills -- not just combat mojo -- matter I think we'll see more folks investing in those feats.

ondali

First Post
In order for a character to make a skill challenge with a 50% chance, its needs to have a 65% chance for each roll.
this corresponds to the fact that you need twice as many success as failures. if you would need 3 times as many success as failures you would need to have a 75% chance of success for each roll to have a 50% chance of making the challenge

Storminator was right in that there are only 2 scenario's that enable a single character to win it, 4 times succes in 4 or rolls or 1 failure in 5 rolls.

The result changes 25.08% to 29,65%.
This doesn't change the fact that I agree with the basic conclusion that a medium skill challenge for a single character are way too difficult.

warning: heavy math

You've got the basic idea of combinations and you are right that C(17 rolls, 12 desired) = 6188
but the calculation should be the sum of
C(17 rolls, 12 desired) = 6188 *.55^12 *.45^5 = 0,087491
C(16 rolls, 12 desired) = 1820 *.55^12 *.45^4 = 0,057184
C(15 rolls, 12 desired) = 455 *.55^12 *.45^3 = 0,031769
C(14 rolls, 12 desired) = 91 *.55^12 *.45^2 = 0,014119
C(13 rolls, 12 desired) = 13 * .55^12 *.45^1 = 0,004482
C(12 rolls, 12 desired) = 1 * .55^12 = 0,000345​

which sums up to results in 19,53% chance of making that challenge
still not really feasible although 5% higher than you're calculations.

The idea of skill challenges works nicely as i've seen it for example in th closing the rift for Kots, where you add some singular easy rolls for a made difficult roll. this gives perfect roleplaying parts.
http://www.wizards.com/default.asp?x=dnd/4dnd/20080522b

Tervin

First Post
Kaodi said:
When I run the numbers, I get ~0.297 for Complexity 1 with 55% Success, and ~0.195 for Complexity 5.

Being a maths teacher I trust my numbers

Complexity 1: 0.256
Complexity 5: 0.147

(Both numbers approximate of course.)

Tervin

First Post
ondali said:
In order for a character to make a skill challenge with a 50% chance, its needs to have a 65% chance for each roll.
this corresponds to the fact that you need twice as many success as failures. if you would need 3 times as many success as failures you would need to have a 75% chance of success for each roll to have a 50% chance of making the challenge

Storminator was right in that there are only 2 scenario's that enable a single character to win it, 4 times succes in 4 or rolls or 1 failure in 5 rolls.

The result changes 25.08% to 29,65%.
This doesn't change the fact that I agree with the basic conclusion that a medium skill challenge for a single character are way too difficult.

warning: heavy math

You've got the basic idea of combinations and you are right that C(17 rolls, 12 desired) = 6188
but the calculation should be the sum of
C(17 rolls, 12 desired) = 6188 *.55^12 *.45^5 = 0,087491
C(16 rolls, 12 desired) = 1820 *.55^12 *.45^4 = 0,057184
C(15 rolls, 12 desired) = 455 *.55^12 *.45^3 = 0,031769
C(14 rolls, 12 desired) = 91 *.55^12 *.45^2 = 0,014119
C(13 rolls, 12 desired) = 13 * .55^12 *.45^1 = 0,004482
C(12 rolls, 12 desired) = 1 * .55^12 = 0,000345​
Sorry, bad math. This way you will count several cases several times. You have to do all 17 rolls for every case to avoid that. (Same thing for the complexity 1 case.)

Edit: Trying to explain why... Your calculations never take into account that the failed rolls in the calculated cases could take place after we had got the 12 needed successes. That is why you end up counting several cases several times.

Also, it is not enough to get twice as many successes as failures. You need more than that. Just barely more than that, but still more than that...

Last edited:

First Post
Where are you getting the extra +5 to the DC from? Reading the DMG it just says to use the difficulty by level table, and that's only 15 at level 1 for a medium challenge. I'm assuming I must be missing something but I can't see anything about a +5.

Replies
9
Views
660
Replies
7
Views
545
Replies
4
Views
617
Replies
8
Views
842
Replies
4
Views
901