D&D 4E Showing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)

Spatula

Explorer
I think what you're seeing in the higher-level DCs being "easier" is the increasing gap between Trained+Prime Score and Trained+Dump Stat and Untrained as the characters level up.

A fighter who is trained in Athletics is going to start with a good bonus because he needs a good strength score. And it will improve at a rate faster than +1/2 lvl because the fighter is improving his strength as he levels up. A +9 bonus at level 1 might be a +28 bonus at level 30.

That same fighter can be trained in Intimidate, probably starting with a middling bonus because he has no particular use for charisma outside of that one skill. But his charisma barely improves with levels (+1 at 11th and 21st lvls). A +6 bonus at level 1 would be a +22 bonus at level 30, and even that is somewhat generous in assuming an above-average stat allocation.

Untrained skills will vary between +23 (better than trained+dump stat) and +15 at level 30. So the DCs have to have more slack built into them to account for non-prime skill uses.

FireLance said:
The math behind skill challenges bothered me too. Then, I looked at the higher-level DCs and noticed something interesting (numbers from pg 42 of the DMG):

For levels 1-3, a moderate check was DC 15 (DC 20 for a moderate skill check).

For levels 28-30, a moderate check was DC 29 (DC 34 for a moderate skill check)

A 30th level character would have a +15 for level, say a +6 from ability scores, a +5 for skill training, and an unknown amount from magic (I haven't digested all the powers and magic items yet), for a total of +26. A trained character would thus succeed 65% or more of the time on an individual skill check, making it more likely that he can overcome a skill challenge without assistance.

So, the skill challenge system for 4e seems to require characters to work together (using Aid Another) to have a decent chance of overcoming the skill challenges at low levels, while allowing them to function more independently at higher levels.

So, what do you think: bug or feature?
 

log in or register to remove this ad

Eldorian

First Post
Very interesting read. Apparently no one at WotC knows what a binomial random variable is. It's obvious from first glance that a none of the standard challenges have better than 50% win rate, cause the mean for a binomial random variable is so easy to compute. If average rolls don't win, it probably isn't a good system.
 

FireLance

Legend
Eldorian said:
Very interesting read. Apparently no one at WotC knows what a binomial random variable is. It's obvious from first glance that a none of the standard challenges have better than 50% win rate, cause the mean for a binomial random variable is so easy to compute. If average rolls don't win, it probably isn't a good system.
What I find truly odd is that the high failure rate wasn't picked up in playtests. Unless, of course, the way to win at skill challenges really is to make full use of the Aid Another action at low levels.
 

Eldorian

First Post
FireLance said:
What I find truly odd is that the high failure rate wasn't picked up in playtests. Unless, of course, the way to win at skill challenges really is to make full use of the Aid Another action at low levels.


The only thing that would explain this to me is that in the playtests they weren't adding +5 to the DCs. If you lower all DCs in a skill challenge where you cannot aid another by 5, then you get average rolls give success. (+9 at first level, so 75% chance of success on a DC 15). Mean number of successes on 6 rolls is then 4, the number required to win. Not entirely accurate math here (as in the OP, only 5 rolls are necessary to consider), but better than default.

Keep on the Shadowfell spoilers:

Reading the keep on the Shadowfell skill challenge tho, yes, that thing is nearly impossible to win, obviously. The Eladrin wizard I made for a game yesterday had only a +11 to arcana, and I thought that was crazy good for first level. He'd stop attempting the skill challenge the first failure he got and the stupid thing attacks him.
 

Eldorian said:
Very interesting read. Apparently no one at WotC knows what a binomial random variable is. It's obvious from first glance that a none of the standard challenges have better than 50% win rate, cause the mean for a binomial random variable is so easy to compute. If average rolls don't win, it probably isn't a good system.
It is not a binomial random variable. It is a negative binomial (or Pascal) variable, which explicitly addresses the calculation of the probability distribution of the number of successes before a given number of failures with basic probability of success p.
I suggest the Original Poster to redo his calculations according to this model.
 

Plane Sailing

Astral Admin - Mwahahaha!
rabindranath72 said:
It is not a binomial random variable. It is a negative binomial (or Pascal) variable, which explicitly addresses the calculation of the probability distribution of the number of successes before a given number of failures with basic probability of success p.
I suggest the Original Poster to redo his calculations according to this model.

Since you apparently have familiarity with it, would you like to take a crack at doing so?
 

Plane Sailing

Astral Admin - Mwahahaha!
FireLance said:
What I find truly odd is that the high failure rate wasn't picked up in playtests. Unless, of course, the way to win at skill challenges really is to make full use of the Aid Another action at low levels.

Although playtesters remain gagged, I get the impression that a large swathe of playtesters were only given very limited subsections of the rules to work with - perhaps just single scenarios - so it may be that most playtesters weren't in a position to actually check this out in depth.

(That strikes me as a bit of a silly way to run a playtest, because your testers are unlikely to unearth real bugs... this skill challenge issue may be a case in point)

Cheers
 

Dyrvom

First Post
Stalker0 said:
Aid Another does exist in a limited form. If its supposed to be the standard in skill challenges, its not explained correctly. If for example, a group of 5 players can have 1 person make rolls, and the other four provide +8 bonuses (which is the example given), then the skill challenge is so easy you might as well not even bother.
The inclusion of a time limit in the vast majority of skill challenges seems like a fundamental of the new system to me. This means aid other would only be used extensively in a crucial climax to a skill challenge, and otherwise would be used sparingly.
 

Eldorian

First Post
rabindranath72 said:
It is not a binomial random variable. It is a negative binomial (or Pascal) variable, which explicitly addresses the calculation of the probability distribution of the number of successes before a given number of failures with basic probability of success p.
I suggest the Original Poster to redo his calculations according to this model.

Dude, I just taught a course on this stuff last semester. You can perfectly model this situation with a binomial random variable exactly as he did. Roll 5 dice at once and see if you get 4 or 5 successes is type 1. If you were to make a tree diagram and continue branches where the experiment technically stops, you'll notice that only the first 5 twigs matter.


edit: LOL. I had no idea what a Pascal variable is, so I wiki'd it. No wonder I didn't know. It's a simple subcase of a binomial.
 
Last edited:

Plane Sailing said:
Since you apparently have familiarity with it, would you like to take a crack at doing so?
I am still waiting for the DMG to arrive, damn Amazon :(
If the DMG assumes a model of "X number of successes before Y number of failures", then the OP's reasoning is wrong.
 

Eldorian said:
Dude, I just taught a course on this stuff last semester. You can perfectly model this situation with a binomial random variable exactly as he did. Roll 5 dice at once and see if you get 4 or 5 successes is type 1. If you were to make a tree diagram and continue branches where the experiment technically stops, you'll notice that only the first 5 twigs matter.
It is a distribution of waiting times, quite different from a binomial distribution. It is the distribution of waiting times in a Bernoulli process.
 

Eldorian

First Post
rabindranath72 said:
It is a distribution of waiting times, quite different from a binomial distribution. It is the distribution of waiting times in a Bernoulli process.

What are you, a statistician? Only a statistician can take simple probability calculations and use some oddly named model to entirely mess it up. The OP's reasoning is perfect. I'd have given him an A. In fact, his reasoning is better than yours, because yours requires you to use the fact that men before you used the OP's reasoning and created this random variable.

PS:

<--- mathematician.
 

Mephistopheles

First Post
Eldorian said:
What are you, a statistician? Only a statistician can take simple probability calculations and use some oddly named model to entirely mess it up. The OP's reasoning is perfect. I'd have given him an A. In fact, his reasoning is better than yours, because yours requires you to use the fact that men before you used the OP's reasoning and created this random variable.

PS:

<--- mathematician.

Them's fightin' words!

I haven't seen talk like that since my days at university. There was an incident in the Mathematics Faculty that resulted in a lost eye, a broken pen, and a ruined pocket protector.
 
Last edited:

Eldorian

First Post
Mephistopheles said:
Them's fightin' words!

I haven't seen talk like that since my days at university. There was an incident in the Mathematics Faculty that resulted in a lost eye, a broken pen, and a ruined pocket protector.


Hey, at my university, those damned statisticians used to have their offices in our building. Luckily, we kicked them out before I really became entrenched here.
 

pemerton

Legend
My maths has been weakened by the passage of years, but assuming a 50/50 chance of success on each check, and just plotting out a probability tree for a Complexity 1 challenge, I get that the odds of success are 6/32, or a bit less than 20% (1/16 of 4 successes straight away, plus 1 chance each in 32 of having the single fail on the 1st, 2nd, 3rd or 4th roll). To bring the overall chance up to 75% is going to require quite a good deal more than a 50% chance on each check.

At 60% chance of success per check, my probability tree with a bit of handwaving to speed things up suggests 1053/3125, or somewhat over 30%, as the overall chance of success for Complexity 1.

At 80% chance of success per check, I make it 2304/3125, or a little under 75%.

An 80% chance of hitting DC 20 requires a bonus of +15 - but at that point it becomes automatic due to taking 10, doesn't it?

If taking 10 is ruled out, then we're looking at +4 from stat, +5 from training, let's say +1 from level (as the DCs are meant to apply to levels 1-3) and then a miscellaneous +5 (+3 focus, +2 circumstance or aid another?).

At level 30, an 80% chance to hit DC 34 requires a +29: let's say +6 from stat, +5 from training, +15 from level and then a miscellaneous +3 (focus, or a magic item and/or a circumstance bonus).

So I'm not sure that the numbers are wrong, just that 1st level PCs seem to be victims of their lack of level, feats and magic items.
 

FireLance

Legend
pemerton said:
So I'm not sure that the numbers are wrong, just that 1st level PCs seem to be victims of their lack of level, feats and magic items.
What I'm deploring is that the lack of feats and magic items has apparently not been taken into account when determining the DCs for low-level characters. ;)
 

Eldorian said:
What are you, a statistician? Only a statistician can take simple probability calculations and use some oddly named model to entirely mess it up. The OP's reasoning is perfect. I'd have given him an A. In fact, his reasoning is better than yours, because yours requires you to use the fact that men before you used the OP's reasoning and created this random variable.

PS:

<--- mathematician.
Ehy, Mr. Funny, I am a mathematician.
But since you are smarter than me, you can explain this:
1) We're interested in the number of failures before reaching a fixed number of successes.
2) The experiment consists of a sequence of independent trials.
3) The probability of success, p, is constant from one trial to another.

We want the probability of x failures before a number of successes r.
This is (Polya/Pascal/negative binomial distribution):
P(X=x|r,p)=OVER(x+r-1, r-1)*p^r*(1-p)^x

For the most basic skill challenge outlined above, x=2, r=4, p=0.55
If you do the math (you are a mathematician, right?), then you get something like:
P(X=2|4,0.55)=0.1240

So, there is only a 12% chance of 2 failures before 4 successes.

Clearly, this result is quite different from what the OP got.

Now, since you are the smarter here, you can either disprove me, or the other guy. I am fine in either case.

Thanks,

EDIT: Oh, obviously one might want the cumulative probability, so:
P(X<=2|4, 0.55)=P(X=0)+P(X=1)+P(X=2)=0.2552

So, the probability of getting AT MOST 2 failures is 0.26
 
Last edited:

vagabundo

Adventurer
You know, I knew there was a reason I hated probability and this thread is IT!!!

Anyway, dont WotC have a full time maths guy working for them? If so did skill challenges slip through or do we need some erratum??

Two points from the OP:

- At 8th (party) level we see the complexity do a flip, and more complex challenges become easier, for 1st level skill challenge. Is this because of the level disparity: does the same thing happen for a 10th level party for a 3rd level skill challenge? If so maybe PCs should auto-win on skill challenges 5 below their level.

- What are people best guesses at the non-broken DCs for skill challenges from the chart (DMG pg42), maybe a +2 for skill checks rather than a +5...
 

two

First Post
rabindranath72 said:
Ehy, Mr. Funny, I am a mathematician.
But since you are smarter than me, you can explain this:
1) We're interested in the number of failures before reaching a fixed number of successes.
2) The experiment consists of a sequence of independent trials.
3) The probability of success, p, is constant from one trial to another.

We want the probability of x failures before a number of successes r.
This is (Polya/Pascal/negative binomial distribution):
P(X=x|r,p)=OVER(x+r-1, r-1)*p^r*(1-p)^x

For the most basic skill challenge outlined above, x=2, r=4, p=0.55
If you do the math (you are a mathematician, right?), then you get something like:
P(X=2|4,0.55)=0.1240

So, there is only a 12% chance of 2 failures before 4 successes.

Clearly, this result is quite different from what the OP got.

Now, since you are the smarter here, you can either disprove me, or the other guy. I am fine in either case.

Thanks,

I love interdepartmental donnybrooks.

I'm on the side of pure math vs. the stats guy.

What about you?

I give pure math 3 to 1 over the Pascal variable.

Taking bets now!
 

pemerton

Legend
FireLance said:
What I'm deploring is that the lack of feats and magic items has apparently not been taken into account when determining the DCs for low-level characters. ;)
Fair enough!

Racial bonuses will presumably make a difference, though - bring on the half-elves!
 

An Advertisement

Advertisement4

Top