D&D 4E Showing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)

[Tervin]

Having done those calculations earlier, I got a hunch for what probabilities would lead to more acceptable results.

I tried setting the probability of success to 70% and then testing for complexity 1-5. Got the following results:

Complexity 1: 0.5282
Complexity 2: 0.5518
Complexity 3: 0.5696
Complexity 4: 0.5842
Complexity 5: 0.5968

This to me would be a decent basis for skill challenge difficulty, where good RP and clever ideas can make the odds even better for the players. So, if the typical skills used in the challenge would be at +9, I would see to it that 14/20 results were successes, in other words DC 16. The suggestion of DC 15 as normal for skill challenges at level 1 will give better chances for success, and is also at a reasonable level.

 

[Wulf Ratbane]

How does that change the math?

Short answer: A lot.

What would be the best way to fix the skill challenges then?

The OP has laid the groundwork. Look back at the first post in this thread for a link to his other thread. There you will see not only his proposed solution, but a discussion of how changing the DCs or the Success/Failure ratio changes things.

Suffice to say the system is not very forgiving of variance in either case.

 

[WhatGravitas]

So the model doesn't care about number of failures, it just cares about when it hits the right number of successes.

Am I wrong?

Unless you do it in reverse, i.e. keep rolling, until you get two failures.

Then use the cumulative probability for the maximum number of rolls you can get without failing (i.e. no. of successes + no. of failures -1) - that's your total chance to fail, hence (1-p) is the chance to succeed.

You can also look this up on the MathWorld site, which spells out the cumulative function... and gives you the function evaluator to get the results, in this case:

p = 0.45 (chance of failing, we're still using it in reverse)
r = number of wanted failures
x = number of successes to win the skill challenge.

(in this order into the linked function evaluator)

Hence we get for a 55%/45% chance the following results:
4/2 = 0.743783 (chance of failing the challenge)
12/6 = 0.852932

-> which are right on your numbers!

Cheers, LT.

 

[Aristeas]

Just to supplement the pure mathematics upthread, I wrote a Java simulator of a complexity 1 skill challenge and ran it ten million times on all of the interesting DCs. This is only an approximation, obviously, but at n=10^7 it's more or less equivalent to the actual probabilities, and you can see it matches what the mathematicians found before.

If the PCs need 3 or better, success chance is 91.85738%
If the PCs need 4 or better, success chance is 83.5259%
If the PCs need 5 or better, success chance is 73.73342%
If the PCs need 6 or better, success chance is 63.29145%
If the PCs need 7 or better, success chance is 52.82564%
If the PCs need 8 or better, success chance is 42.84524%
If the PCs need 9 or better, success chance is 33.70783%
If the PCs need 10 or better, success chance is 25.62302%
If the PCs need 11 or better, success chance is 18.73536%
If the PCs need 12 or better, success chance is 13.12077%
If the PCs need 13 or better, success chance is 8.71162%
If the PCs need 15 or better, success chance is 3.06186%
If the PCs need 16 or better, success chance is 1.56657%

So, as written, if our 1st level PCs somehow manage to have an average +10 to their checks, they have a 26% chance on moderate tasks, a 74% chance on easy tasks, and a 3% chance on hard tasks. If we drop the +5 DC, moderate tasks are 74%, and easy tasks are better than 90%, while hard tasks are still only 26%.

 

[Ydars]

The DMG points out that the use of Rituals and powers may bring automatic successes during skill challanges, which would increase the probabilities of getting an overall success, yet none of the examples use this feature.

 

[DSRilk]

I like math, but for me, it's far quicker and more accurate to simply make a computer simulate it and give me the results. I did just that (you can find it at http://www.ebonterr.com/4e/skill_challenge_test.htm). I wrote it to run 10,000 runs of diff 1 - 5 skill challenges. It assumes moderate difficulty level 1 skill checks (DC 20) and an optimal character making each check with a skill of +9 (18 ability score with a trained skill). Note that all the params I used can be modified on the page above, so you can run tests with whatever numbers you want to try out.

As a note, reading the rules on skill challenges in the DMG plus the example, while it's possible to give others a +2 bonus on a check, it's not that common. You don't just use aid another. The one example in the DMG where you can give someone +2 actually comes from something written in the challenge where a character unearths information on the duke and makes a successful history check.

Here are the results for diff 1 - 5 challenges at level 1 (results have varied by as much as 1%):

% success

diff 1: 18.92 %
diff 2: 14.3 %
diff 3: 11.09 %
diff 4: 9.21 %
diff 5: 7.08 %

 

[GruTheWanderer]

I like math, but for me, it's far quicker and more accurate to simply make a computer simulate it and give me the results. I did just that (you can find it at http://www.ebonterr.com/4e/skill_challenge_test.htm).

Well done!

 

[Tarril Wolfeye]

It's the footnote to the DC table on page 42. I didn't notice it at first, either.

I think the +5 is only for singular skill checks, NOT for skill challenges. Does the math work if this is the case?

 

[DSRilk]

Yes and no

% chance of success

diff 1: 63 %
diff 2: 67.97 %
diff 3: 71.01 %
diff 4: 73.89 %
diff 5: 76.3 %

So "yes" it's a more reasonable chance of success...

but "no" it's still irrational that higher difficulty challenges end up being notably easier than low difficulty ones.

 

[2WS-Steve]

or how about keeping the DCs but changing the required successes? So, instead of 4 successes before 2 failures, how about 4 successes before 4 failures? How does that change the math?

This would be my solution.

Essentially, its the World Series way of doing things -- first team to 4 wins takes the series. You can imagine how unfair it would be to require one team to win 4 timers and the other team only have to win 2 times -- well, that's the way the rules as written work.

If you go first-to-4 (or any other number) then, just like in the World Series, you're reducing the effect of randomness -- making the process more stable. So, if you are already likely to succeed (you only need to roll a 10 or less), you'll be more likely to succeed the higher the number of successes/failures required is. Contrariwise, if you're not so good at the check and need to roll a 12 or higher, more "games" in the series means you have a lower overall chance of success.

This similarly applies to the rules as written -- but the break point isn't at 50% chance of success on an individual check. Instead it's at 66% chance of success on an individual check. If you need to roll a 6 or less then higher complexity tasks are in your favor since, in the long run, individual flukes of bad luck will get drowned out.