log in or register to remove this ad

 

4E Showing the Math: Proving that 4e’s Skill Challenge system is broken (math heavy)

2WS-Steve

First Post
Plane Sailing said:
Don't misunderstand me - I'm not laying this at the feet of the playtesters at all!

My concern is that WotC probably played their cards so tight to the chest that of the dozens of playtest groups, very few actually had even the opportunity to test this. I think that many of the playtest groups were probably testing scenarios rather than the rules.

Cheers

I also wouldn't be surprised if this was out there but just never got used. These "make a bunch of rolls" skill systems have been around for a while in other games and always look much better on paper than they play in real life.

At the table you just change one die rill into several die rolls. That gets tedious.

What makes combat fun isn't just the die rolling, but the fact that every die roll is the result of a decision the player or DM makes -- and it's seeing the results of your choices that makes it interesting.

But the skill challenge system has, at best, one tiny decision: what skill do I roll? And that's an easy decision -- the one with the highest bonus that is relevant -- so it's barely a decision at all.

Players and DMs realize how boring this is quickly, and then don't bother with the challenge system in the future -- thus it never gets really play-tested. Even moreso if the challenge system proves to usually result in failure for some reason of mysterious math.

But the rest of the game works really great and is very fun, so people just ignore the parts that need more thought.
 

log in or register to remove this ad

Larry Hunsaker

First Post
I do not have my DMG yet, but I think I will define failure in a different way. Not sure if the skill challenge system defines failure as a non-success, but like 3E I would probably have 3 possible results, success, non-success and failure. Failure would be failing the DC by 5 or more. This would result in an instant failure. If you simply got a non-success, that is you failed but by less than 5, then that skill would have its future check DC increased by 5, but that non-success would not count as a failure. This would give a chance to redeem the non-success, but making it a tough one. Once you succeed in that skill, the +5 DC penalty is removed until another non-success results. So if you fail a check by 4 or less, you do not succeed nor fail, but if you try that same skill again, you have a +5 DC now, and that same failure by 4 or less would then result in a true failure, because the +5 DC would make that a fail by 9 to 5. This would alert the PCs that a particular skill has been pushed to its limit and further uses may be risky, and it lets lower skill PCs try it once with some cushion for a slight miss.

You could boost this DC failure to a failure by 10 or more to make it more forgiving, perhaps this would apply to skills that the Player who used the made an excellent role-play attempt, so you give him a deeper cushion of failure, giving more incentive to role-play.
 

DSRilk

First Post
I updated http://www.ebonterr.com/4e/skill_challenge_test.htm so that you can now specify the average char skill value, the challenge dc, as well as the number of required successes and max failures so that you can test virtually all scenarios related to skill challenges and find what works best for what you want to accomplish with a challenge. To work out things like easy checks that might give +2 bonuses, just figure out how often they're likely to occur. Just once? Then basically ignore it. If the chars could get that bonus for about half the checks, add 1 to the char skill value. If they could get it for every check, increase the char skill value by 2. Enjoy! And yes, the UI is ugly -- I won't use it enough to make it pretty ;)
 

Larry Hunsaker

First Post
Another idea I am toying with is to use the system as is, but to allow for other PCs to aid the primary PC in each skill check made. The primary PC is the one who makes the check. If he fails, the other PCs can then try to assist him to recover from failure. This is a retroactive aid other, and the DC to make it would be 5 lower than the primary check DC. If the first assistance makes the reduced check, he gives a +2 bonus to the primary PC check, and if this is not enough to succeed, the next PC can try. But this cascade of skill assists stops once the first PC assistant fails, after that, no additional help can be offered. So as long as the assists succeed, the primary PC keeps racking up +2 bonuses and perhaps (if he did not fail by too much) his initial failure may become a success thanks to the help of the party.
 

silentounce

First Post
DSRilk said:
I updated http://www.ebonterr.com/4e/skill_challenge_test.htm so that you can now specify the average char skill value, the challenge dc, as well as the number of required successes and max failures so that you can test virtually all scenarios related to skill challenges and find what works best for what you want to accomplish with a challenge. To work out things like easy checks that might give +2 bonuses, just figure out how often they're likely to occur. Just once? Then basically ignore it. If the chars could get that bonus for about half the checks, add 1 to the char skill value. If they could get it for every check, increase the char skill value by 2. Enjoy! And yes, the UI is ugly -- I won't use it enough to make it pretty ;)

The fact that we require something like this to determine how to modify the RAW so that the PCs can actually succeed at a decent, and fun for the game, rate pretty much proves that this system is broken. Honestly, I doubt they'll do anything about it, because there's not an easy or quick fix to make it less wonky. It'll be interesting to see how, if it all, WoTC addresses this in an errata or reprinting. The fact that the probabilities already existed in a previously published supplement and they still messed it up is somewhat amusing and disturbing at the same time. Surely, they would have used that for a reference.
 
Last edited:

DSRilk

First Post
...pretty much proves that this system is broken...

yes, it is

...there's not an easy or quick fix to make it less wonky...

yes, there is -- change the # success = 2 x failures to something more reasonable -- like X successes or X failures. Play around with the page I made and you'll see there are several easy ways to fix this.
 

silentounce

First Post
Can you make one change like that will work in a consistent manner for parties of different levels like the system is supposed to work? Judging from comments in here and in the thread that the OP linked to, probably not. I'd rather not have to go to your webpage and plug in a bunch of info every time and tweak it in order to get an appropriate challenge for my party. That's not a quick fix.

One of the big problems is the swinginess/variance, and simply changing the number of successes to something more reasonable doesn't fix that.
 

DM_Blake

First Post
It looks like all the math in this thread points to the RAW chance of success being in the teens, or twenties. So, around 20%, plus or minus for situational mondifiers and mathimatical variations.

As for me, I would prefer the chance to fail was around 20%

Therefore, I'm pretty sure what I'll do is simply reverse the success to failure ratio. For example, instead of 4 successes before 2 failures, I will probably go with 2 successes before 4 failures.

That way I can leave the RAW completely alone. I can stay with the DCs as presented and use the examples provided. All I have to do is swap the number of successes for the number of failures.

This way I will even be able to use any officially published dungeons without having to tweak DCs or muck with the challenge in any concrete way.

The only thing I don't like about swapping the ratio is that it will result in many challenges being over with just a couple skill rolls. It won't involve the whole party. So it's likely that I will take most skill challenges that only require 2 or 3 successes and I will double (or something like double) both the number of successes and the number of failures (which, interestingly enough, works out pretty much the same as just quadrupling the original number of failures without swapping the ratio - which may also end up being what I do much of the time).
 

Tervin

First Post
DM_Blake said:
It looks like all the math in this thread points to the RAW chance of success being in the teens, or twenties. So, around 20%, plus or minus for situational mondifiers and mathimatical variations.

As for me, I would prefer the chance to fail was around 20%

Therefore, I'm pretty sure what I'll do is simply reverse the success to failure ratio. For example, instead of 4 successes before 2 failures, I will probably go with 2 successes before 4 failures.

I would not recommend that. It will lead to even weirder results, where higher complexity leading to over 90% and probably be even harder to get good numbers out of.

I set up a simple Excel sheet for calculating skill challenge probabilities, and the data from that says that you should set the DCs so that the players on average need to roll 5 or higher.

That gets the following results:
Complexity 1 0.73728
Complexity 2 0.79692
Complexity 3 0.83886
Complexity 4 0.87016
Complexity 5 0.89430

Attaching the sheet in case somebody wants to play with it. Regrettably it can't handle skill challenges outside the five normal complexities - that would make the formulas a lot more complicated...

(If the sheet looks weird it could be that it was made using a Swedish version of OpenOfficeCalc...)
 

Attachments

  • skillchall.xls
    64.5 KB · Views: 25

LostSoul

Adventurer
DSRilk said:
but "no" it's still irrational that higher difficulty challenges end up being notably easier than low difficulty ones.

Except that higher complexity challenges aren't supposed to be more difficult:

"Set the complexity based on how significant you want the challenge to be."

Complexity is about real-world handling time, not difficulty.
 

seusomon

Explorer
Very interesting thread. I don't have my DMG yet, but it sounds to me like the main problem is the expectations set up in the text about the difficulty of a typical skill challenge. The system itself is interesting, and could be fun for players with the right sort of mindset to milk it for roleplaying.

I'd like to suggest that DMs think about skill challenges this way:

If you require more successes than you allow failures (the RAW state twice as many), then what this represents is a very delicate situation, a challenge that must be handled nearly without error - a small number of slips, and you lose. These challenges are much more difficult than a basic skill check, and become more difficult as the complexity increases (barring really superior probability of success on each check).

If you require an equal number of successes as you allow failures, then the challenge is roughly as difficult as a single check would be - the difference is just that there is more detail and back-and-forth involved.

If you require fewer successes than you allow failures, this represents a "forgiving" situation - one in which the PCs can mess up a fair amount, as long as they get something to work before an inordinate amount of interaction has taken place.

As a DM, I will use the system but make adjustments to the failure/success ratio according to the thinking above. There is a role for skill challenges of the sort described in the rules, but that role is a narrower niche than was apparently implied in the text. It's not like an "average encounter" - more like an encounter that can only be survived if all the pieces come together just right.

Modifying the DC of the checks or giving the PCs bonuses is more of a band-aid solution - as more and more checks are made, the math of the probabilities will eventually drive the challenge toward failure unless the bonuses are awfully good.

Finally, a point about the more complex challenges actually being easier once the probability of success exceeds a critical level. This makes sense when failures are rare, because then repetition is on your side, smoothing out the effects of the occasional fluke failure. In game world terms, you could imagine a very skilled diplomat, for example, who actually wants the negotiations to be long and extended, because then he has the opportunity to bring all his skills to bear and to correct any missteps he might have made as the negotiations started off. If you have only modest skill at something, your hope is for luck to help you out, and it's not to your advantage to be repeatedly tested. If you are highly skilled, then luck is more an enemy than an ally - you want to keep at it and rely on your talent.
 
Last edited:

Nail

First Post
Great Thread! Thanks everyone. This is the reason I love ENWorld.

But....boy does this suck. Really. Higher success rate with higher complxity? Sucks. Just sucks.

I find it *really* amusing that our simulations about how Skill Challenges where going to work (pre-May 25th) are actually much better than how they actually work. :lol: Crazy, man. Open source projects rule.
 

TwinBahamut

First Post
This thread is rather depressing...

People keep going around saying essentially "our quick five-minute analysis is perfect, so obviously WotC's several months of detailed design and playtesting was a total joke". Meanwhile, attempts to show that the core assumptions of the five-minute analysis might be flawed (the +5 rule) are shot down without detailed examination.

In other words, people in this thread are more eager to prove that WotC screwed up than try to see how the rules may actually work and how they can actually work with a different interpretation of the text.

If you are supposed to add +5 to everything on the chart, then why wasn't the +5 already added into the chart in the first place? That would be far more consistent with 4E's design, after all. The entire point is to remove the need for the DM to do a lot of work in order to get a proper result. Unless you can resolve that contradiction, then the OP's core assumptions are demonstrably false, meaning his conclusion is false.

It is probably better to assume that seeming exceptions to the general rule that there is a 70% or so success rate should be treated as just that: exceptions, not indicators of a hidden general rule.
 

NewfieDave

First Post
DMG said:
Set a level for the challenge and DCs for the checks involved. As a starting point, set the level of the challenge to the level of the party, and use moderate DCs for the skill checks (See the Difficulty Class and Damage by Level table on page 42).

As per the referenced table on page 42 (which is the best thing the DMG has EVER given me!), the DC for a level 1 skill check of moderate difficulty is 15. I haven't got the slightest clue where you found that extra +5.
 

Nail

First Post
TwinBahamut said:
If you are supposed to add +5 to everything on the chart, then why wasn't the +5 already added into the chart in the first place?
Read page 42. The reason becomes evident if you read the section. Try that before posting to avoid embarrassment.


The DCs presented in the chart are for attacks. That's why the +5 for skill checks are not added.
 

silentounce

First Post
TwinBahamut said:
This thread is rather depressing...

People keep going around saying essentially "our quick five-minute analysis is perfect, so obviously WotC's several months of detailed design and playtesting was a total joke". Meanwhile, attempts to show that the core assumptions of the five-minute analysis might be flawed (the +5 rule) are shot down without detailed examination.

In other words, people in this thread are more eager to prove that WotC screwed up than try to see how the rules may actually work and how they can actually work with a different interpretation of the text.

If you are supposed to add +5 to everything on the chart, then why wasn't the +5 already added into the chart in the first place?
That would be far more consistent with 4E's design, after all. The entire point is to remove the need for the DM to do a lot of work in order to get a proper result. Unless you can resolve that contradiction, then the OP's core assumptions are demonstrably false, meaning his conclusion is false.

It is probably better to assume that seeming exceptions to the general rule that there is a 70% or so success rate should be treated as just that: exceptions, not indicators of a hidden general rule.

Two questions, have you read this ENTIRE thread, and have you read all the pertinent sections of the DMG(Chapters 3 and 5 for the most part)? Because most of what you are complaining about has been addressed in this thread already, especailly that +5 thing. Read what I bolded in your post above, now go look at the table on page 42 and read what is below the line for 28th-30th levels. Read the example on page 42 and look at the bottom of the chart, it's clear as day. How is that a contradiction?

Everything about DCs in the skill challenge section refers to that table. And that table explicitly states that skill checks get a +5 DC. Not to mention that even when not boosting the DC by 5 this system is still broken. Which was already discussed earlier in this thread.

NewfieDave said:
As per the referenced table on page 42 (which is the best thing the DMG has EVER given me!), the DC for a level 1 skill check of moderate difficulty is 15. I haven't got the slightest clue where you found that extra +5.

It helps to read the entire table. As I mentioned above, there are too very important notes below the table. And the default of that table is for ability checks, not for attacks as the guy above me said, which is why there's a +2 for attacks note.
 
Last edited:

Stalker0

Legend
I appreciate a lot of the compliments I have received for starting this post, but let me say that I wasn't the first person to reveal this problem, there were threads on it before I even knew there was a problem.

However, if you would like to give me praise, please follow the link in my first post. This is a solution to the skill challenge problem. It sets the win rate at a good number, it lowers the variability between complexities, it reduces the variation with changing skill DCs, it fixes the problem that higher complexities can give easier results, and its more interactive and dynamic then the original system. I have spent hours and hours going over calculations and simulations to make it work, and I'm very proud of it.

Further, its only getting better! I'm already working on a new simple and elegant rule to reduce the variance even more. With luck, the new system will be absolutely rock solid. So please take a look.

People are asking the question, can you simply use the numbers in the table, don't add the +5, and will the system work?

The answer is...kind of. The win rate becomes much more reasonable, but you still have the complexity inversion problem, and the system is HORRIBLY intolerant of variation. The difference in win rate between complexity 1 and 5 is huge. The difference for a party with a 65% chance of beating each skill and a 60% is large. Its just not a good system either way.
 

diamabel75

First Post
It seems that a better way (assuming that the +5 pertains to individual skill checks and not those associated with a skill challenge) would be to have a static 4 failures and simply increase the successes needed to increase the 'complexity' of the challenge.
 

Mal Malenkirk

First Post
Weird.

I have been using 10/15/20 as my checks and it's worked very smooth. In truth, I've also used a lot of intermediary numbers like 12.

Where did I get this from? Anyway, it's fairly intuitive. 10 is close to a sure thing but not quite; all it takes is one bad roll + 1 PC with no immediately useful skills and you're on edge on a complexity 1 challenge. 20 is really rough, you better have a plan for what to do if they fail, which they probably will, and have a cool reward for success. At a glance I would expect 25 to be so statistically unlikely that there is no point attempting it. 15 is balanced.

Frankly, I just eyeball it, the way you estimate your odds at the poker table, you know? Worked very fine so far.
 

silentounce

First Post
diamabel75 said:
It seems that a better way (assuming that the +5 pertains to individual skill checks and not those associated with a skill challenge) would be to have a static 4 failures and simply increase the successes needed to increase the 'complexity' of the challenge.

But that's not just increasing "complexity", that's also decreasing difficulty. Which is one of the problems of the RAW to begin with.
 

Advertisement2

Advertisement4

Top