D&D 5E Modeling Uncertainty

Bawylie · Jan 19, 2017

BookBarbarian said:
I usually do a similar preamble, though I find my players still seem to slip back. They may just be special, or we might just not be playing enough for it to sink in.

I try to take off an article of clothing or two during the preamble. Spice things up, keep attention and focus on me.

-Brad

Saeviomagy · Jan 19, 2017

Elfcrusher said:
Yes. Not at all the kind of situation I'm describing.

How not? If I make a one-shot device, I don't know until I trigger it whether it will work. I think it will, but it's not a guarantee. It sounds like exactly the scenario you are putting forward.

Cool, so even the 1 in 1,000 sort of lie detecting savant (I haven't read the actual research so I'll take it on face value) only has an 80% chance of success. Totally confirms why this system makes sense.

80% is the number because they pre-selected that as the threshold for truth-wizardry. Some of the candidates had higher scores.

Incidentally, 80% would be equivalent to having a +7 insight check if we assume that the average modifier of the liars they were tested against is +0.

Funnily enough, the general population had a success rate of 54% which pretty much mirrors a +0 bonus vs an average of a +0 bonus.

None of them had a 100% success rate. All this tells us is that none of the truth wizards had a modifier that was (in the case of an opposed roll) 19 points higher than their opponents. Since I can't find the raw data of the study, I can't guess what the highest modifier actually was. I think it would be reasonable to assume that given the sampling the random spread of deception bonuses would match the random spread of insight bonuses, and therefore it would be impossible for anyone to score 100%.

Ok, I wasn't specific enough about the scenario. I was thinking of the movie version, where you've got 7 seconds to decide, and no time to work through the schematic.

Then there's no skill check involved at all. I don't know why you're making up a system to handle this at all.

If you're talking about some intermediate point where you don't have enough time to adequately analyse the circuit, then you're talking about disadvantage or worse DCs. The character's uncertainty comes because they don't know the DC.

The simple way to do that is simply to not use fixed DCs. The character rolls, the device rolls and the player has to assess his success based only on one half of the situation and then presumably cut the wire or not depending on how he feels about that. Well, actually simply not announcing the DC works too.

Also in this specific scenario, you could roll the check at the point of the wire cutting, so the PC is deciding solely on how good he is at the task in question. That does leave the PC without any interesting options to mitigate the effect of a poor roll though.

The only way I see this scenario as being interesting is this:
1. present it.
2. player makes a roll to see if he knows which wire to cut. Whether his roll is good or bad, he isn't likely to be confident in the answer unless you've done something like told him the DC up front.
3. optionally the DM rolls an opposed check to add some extra uncertainty, which could cause the bomb to fail despite the player botching his roll, or cause the bomb to explode despite him doing well at it.

I'm not sure how your extra rule would help here. If the PC is only just beating the DC of the device, there's already uncertainty. Heck, if your narrative is strong, he's not going to be confident that he has the device beat because he doesn't know the DC.

As long as everybody understands the rule then a roll that beats the DC is not succeeding. Yet.

Well, as long as they understand and don't object to the whole "being good at something makes you also worse at it sometimes".

Ah, now I see what you are saying. Yes the person who is more likely to succeed is therefore also more likely to be wrong. The easiest way to demonstrate that is by pointing out that the guy with no chance to succeed has no chance to get the wrong answer; at worst he will get no answer.

I referred to this earlier; ideally I would like a failing roll to also have the possibility of becoming a false answer, but I don't see how that's possible if the player is allowed to roll their own dice, which was one of my design goals.

So just roll opposed checks or don't reveal the DC. All you need is a source of uncertainty.

At the same time, while this result is counterintuitive I don't think it really affects gameplay. While those with higher skill are more likely to get an incorrect answer because they succeed more, the proportion of their successes which are false successes will be lower. It's the absence of the "false failures" that makes it look wonky, but it doesn't create an incentive to keep your skill low, or not attempt things. It's sort of analogous (philosophically if not mathematically) to saying that Mitt Romney pays less income tax than his secretary. No, he pays several orders of magnitude more tax; it's his tax rate that's lower.

It makes investing in skills that are likely to have this issue a poor deal, and attempting to use them even more fraught than they already are. I may as well just not use the skill and assume the worst. The fact that you seem to have tarred every insight roll with this brush seems ominous.

OK, here I thought you were saying that the DM would have to provide the information that leads to the incorrect conclusion. I.e., "the guard is visibly sweating." But maybe you're just saying that if the correct answer is 42, the DM is going to have to pick a different, false number to provide.

Can you give me a more complex scenario? The scenarios I'm thinking of have discrete and obvious options: if the answer is "snip the red wire" then the false success is "snip the blue wire". If the answer is "take the left passage" then the false success is "take the right passage". I suppose if there are more wires or passages then the DM will have to pick one, but that shouldn't be too taxing.

No, I'm providing some information as to why the PC is concluding what he is concluding. If the answer is literally "it's just a hunch, you have no information to base your decision on" then why is it a skill check at all? Just randomly pick the answer and tell them it's a hunch.

Also, let me repeat again that I'm proposing this only for specific scenarios, where the character would be going off of intuition or hunch, not hard knowledge.

Again, if that's the case, why does the skill apply to it at all? And why do all insight checks fall into the "just a hunch" bucket?

If the player is rolling History to remember the nickname of a long-dead Archduke then I probably wouldn't invoke this rule, so I wouldn't need an incorrect nickname queued up.

How is this a different scenario to using disable trap on the 3 wire bomb? Or using one's skills to determine if someone is lying? Why is it impossible to get the duke's nickname wrong?

Guest 6801328 · Jan 20, 2017

Saeviomagy said:
How not? If I make a one-shot device, I don't know until I trigger it whether it will work. I think it will, but it's not a guarantee. It sounds like exactly the scenario you are putting forward.

Because it's the product of study and specialized knowledge, not intuition and hunches.

Or, to dive a little bit deeper, if you make a mistake building a bomb it's because you made a mistake. And the dice roll says you didn't make a mistake. The underlying principles are not bayesian or stochastic.

But if you're trying to intuit whether a guard is lying, even if you've been foolish enough to pay for a course in how to detect lies you're still applying probabilities, not certainties. "When people behave such-and-such a way it probably means they are lying."

So even if you apply those principles without making a mistake they still aren't guaranteed to produce the right answer.

How's that?

If you're talking about some intermediate point where you don't have enough time to adequately analyse the circuit, then you're talking about disadvantage or worse DCs. The character's uncertainty comes because they don't know the DC.

That reduces the chances, and also reduces the character's confidence that they will succeed, but has no change on their confidence that they have succeeded. Again, you may not want that sort of uncertainty in your game (which is understandable because it's not, as far as I'm aware, baked into any RPG) but I do, which is why I started the thread.

The simple way to do that is simply to not use fixed DCs. The character rolls, the device rolls and the player has to assess his success based only on one half of the situation and then presumably cut the wire or not depending on how he feels about that. Well, actually simply not announcing the DC works too.

Yes, there are various ways to introduce the kind of uncertainty I'm talking about; the difference becomes the probabilities that are generated.

Also in this specific scenario, you could roll the check at the point of the wire cutting, so the PC is deciding solely on how good he is at the task in question. That does leave the PC without any interesting options to mitigate the effect of a poor roll though.

The only way I see this scenario as being interesting is this:
1. present it.
2. player makes a roll to see if he knows which wire to cut. Whether his roll is good or bad, he isn't likely to be confident in the answer unless you've done something like told him the DC up front.
3. optionally the DM rolls an opposed check to add some extra uncertainty, which could cause the bomb to fail despite the player botching his roll, or cause the bomb to explode despite him doing well at it.

I'm not sure how your extra rule would help here. If the PC is only just beating the DC of the device, there's already uncertainty. Heck, if your narrative is strong, he's not going to be confident that he has the device beat because he doesn't know the DC.

Well, as long as they understand and don't object to the whole "being good at something makes you also worse at it sometimes".

As noted previously that's an erroneous characterization. If you are good at something then on any given attempt you still have a higher probability of succeeding.

So just roll opposed checks or don't reveal the DC. All you need is a source of uncertainty.

Almost. There are different kinds of uncertainty.

It makes investing in skills that are likely to have this issue a poor deal, and attempting to use them even more fraught than they already are. I may as well just not use the skill and assume the worst. The fact that you seem to have tarred every insight roll with this brush seems ominous.

Seriously? Choosing Insight as one of your skills suddenly becomes a bad investment? Sheesh.

No, I'm providing some information as to why the PC is concluding what he is concluding. If the answer is literally "it's just a hunch, you have no information to base your decision on" then why is it a skill check at all? Just randomly pick the answer and tell them it's a hunch.

Oh, ok, so it was the first version. I don't know why you think this is hard. Remember that the fluff in this case is purely fluff. "You think he's lying because he won't make eye contact." "You think it's the left passage because you think you feel a slight draught." "You think it's the red wire because your bomb disposal instructor told you that 66% of the time it's the red wire." Whatever. Unlike the approach where the DM is trying to actually convey the probability with the rationale, in this case the believability of the rationale has no impact on the player's understanding of the probabilities.

DM: "You think it's the red wire because last night you had a dream where everything was red."
Player: "Yeah, whatever. I beat the TN by 7 so I have a 1/6 chance of being wrong."

Again, if that's the case, why does the skill apply to it at all? And why do all insight checks fall into the "just a hunch" bucket?

How is this a different scenario to using disable trap on the 3 wire bomb? Or using one's skills to determine if someone is lying? Why is it impossible to get the duke's nickname wrong?

(Funny you mentioned 3 wires...see my next post, which is what I logged on to write.)

It's not impossible to get the duke's nickname wrong. The difference is that thinking somebody lying is a hunch (see above) not a retrievable fact. I mean, the duke's nickname COULD be a hunch and if, depending on the scenario, the DM decided that uncertainty would add to dramatic tension, this rule could still be used. Similarly (or conversely), whether or not the guard is lying might not really have much impact on the story, in which case I wouldn't bother with the uncertainty.

Why do I sense so much hostility from you toward this whole idea? Or is it hostility toward me personally?

Guest 6801328 · Jan 20, 2017

So, I thought of another way to use skill rolls to reduce uncertainty, but not eliminate it, in a way that may be more acceptable. It's not useful in all situations, and specifically not ones with binary outcomes (the guard is either lying or he isn't), but when it can be used I think I really like this one:

Give three options....three wires, three tunnels, three buttons, three nicknames of the duke, whatever...and if the player succeeds at their skill test you eliminate one of the false ones.

There: the odds have just improved, but there's no extra dice, no whining about "but I was successful", no debates about mathematics. (Well, unless somebody starts claiming it's identical to the Monty Haul problem. It's not.)

You could also have more than three options and then eliminate as many options as you like. E.g., 5 wires down to 2 wires improves their chances by 250%.

Saeviomagy · Jan 23, 2017

Elfcrusher said:
Because it's the product of study and specialized knowledge, not intuition and hunches.

Or, to dive a little bit deeper, if you make a mistake building a bomb it's because you made a mistake.

That sentence is completely redundant. If a bomb that you made fails, it's not always because you made a mistake. The same applies to any engineering product. If you've made something bespoke, then the odds of it working perfectly the first time are never 100%.

And the dice roll says you didn't make a mistake. The underlying principles are not bayesian or stochastic.

The dice roll says simply that random chance says you failed/succeeded. Always putting that random chance down to "this time the PC

ed up" makes most games into clown shows, because 5e has such a high chance for a competent individual to fail. I much prefer "this time something external happened that made the scenario more difficult". If you screw up making a bomb, then one of the components was bad and you didn't catch it. It got jarred in transit. The day was especially humid etc etc.

But if you're trying to intuit whether a guard is lying, even if you've been foolish enough to pay for a course in how to detect lies you're still applying probabilities, not certainties. "When people behave such-and-such a way it probably means they are lying."

Nothing is certain. Have you heard of an engineering quality called tolerance? Do you understand the procedures of QA? You make things and then you test them until you're statistically certain that your product is good.

In D&D terms, what you are describing with respect to is the guard lying is adequately modeled by the opposed check and the fact that you do not know how many ranks he has in deceive.

So even if you apply those principles without making a mistake they still aren't guaranteed to produce the right answer.

Just like engineering you mean?

How's that?

Wrong on multiple fronts, like I pointed out.

That reduces the chances, and also reduces the character's confidence that they will succeed, but has no change on their confidence that they have succeeded. Again, you may not want that sort of uncertainty in your game (which is understandable because it's not, as far as I'm aware, baked into any RPG) but I do, which is why I started the thread.

Like I said: if the player doesn't know the DC, then lowering their final roll will make them more worried that they've failed. Also it IS baked into RPGs, it's called an opposed roll. You've not come up with a clever new mechanic here. Any game that has the DM roll part of a check can leave the player without certainty. That's all that you are doing.

Yes, there are various ways to introduce the kind of uncertainty I'm talking about; the difference becomes the probabilities that are generated.

Other methods have the advantage of being well designed.

As noted previously that's an erroneous characterization. If you are good at something then on any given attempt you still have a higher probability of succeeding.

The guy who is terrible at a roll has a lower chance of screwing up completely than the guy who is great at it. That makes this a bad mechanic, especially when alternative methods exist to do the same thing which don't have the same drawback.

Almost. There are different kinds of uncertainty.

Sure, but yours isn't some special kind except for it's reversal of expectation.

Seriously? Choosing Insight as one of your skills suddenly becomes a bad investment? Sheesh.

Well mostly because you seem set on the idea that any attempt to divine truth is "just a hunch". You seem to have a great lack of understanding of the topic while at the same time asserting that the rules that exist are unrealistic. A quick perusal of the study I cited suggests that of all the skills in the game, insight may be one of the few that actually is a good model of reality.

Oh, ok, so it was the first version. I don't know why you think this is hard. Remember that the fluff in this case is purely fluff. "You think he's lying because he won't make eye contact." "You think it's the left passage because you think you feel a slight draught." "You think it's the red wire because your bomb disposal instructor told you that 66% of the time it's the red wire." Whatever. Unlike the approach where the DM is trying to actually convey the probability with the rationale, in this case the believability of the rationale has no impact on the player's understanding of the probabilities.

You're just assuming that your players aren't listening to you and thinking about what you say, aren't you? They all sound like hedges.

DM: "You think it's the red wire because last night you had a dream where everything was red."
Player: "Yeah, whatever. I beat the TN by 7 so I have a 1/6 chance of being wrong."

Does this not show to you how your system is stupid?

It's not impossible to get the duke's nickname wrong. The difference is that thinking somebody lying is a hunch (see above) not a retrievable fact.

Again - scientific study says you are wrong. If you were correct, the study and follow up studies would not have been able to find people who could consistently detect lies, and would not have found that the general population has a 54% chance to detect lies.

I mean, the duke's nickname COULD be a hunch and if, depending on the scenario, the DM decided that uncertainty would add to dramatic tension, this rule could still be used. Similarly (or conversely), whether or not the guard is lying might not really have much impact on the story, in which case I wouldn't bother with the uncertainty.

Why do I sense so much hostility from you toward this whole idea? Or is it hostility toward me personally?

Why would I be hostile towards you personally? I have no idea who you are.

My hostility is against an idea which
1) Is being created for no reason: mechanics for what you want to achieve already exist.
2) Has been admitted by you to be less than ideal, even when applied in ideal situations. Statistically IS less than ideal.
3) Is applied only to skills that the DM thinks of as unscientific. Which ones? Who knows!
4) Is specifically intended for a skill which actually lines up extremely well with the existing rules.
5) Appears to be intended for scenarios where proficiencies should not apply.

Guest 6801328 · Jan 23, 2017

Saeviomagy said:
That sentence is completely redundant. If a bomb that you made fails, it's not always because you made a mistake. The same applies to any engineering product. If you've made something bespoke, then the odds of it working perfectly the first time are never 100%.

Well, if you can't see the difference I'm not sure I can help you.

Nothing is certain. Have you heard of an engineering quality called tolerance? Do you understand the procedures of QA? You make things and then you test them until you're statistically certain that your product is good.

Oh, yikes. If you want an RPG that models 6 Sigma I think you'll probably have to write it yourself. (And probably play it yourself.)

But take a gold star for pedantry, anyway.

You've not come up with a clever new mechanic here. Any game that has the DM roll part of a check can leave the player without certainty. That's all that you are doing.

Uhhh....yeah. I'm not claiming to be clever or new, was just asking what people thought of it. And upstream I pretty clearly said that an opposed roll can sort of accomplish the same thing as long as it's secret; the only difference is what probabilities result, and how aware the players are of those probabilities. (And also that there are cases (i.e. choosing between passages) where it's not obvious who or what is rolling which opposed skill.)

I'm sorry that you don't seem to understand that my system, as flawed as it may be, produces different probabilities.

Other methods have the advantage of being well designed.

Slave, thou hast slain me! Villain, take my purse: if ever thou wilt thrive, bury my body, and give the letters which thou find’st about me to Edmund earl of Gloucester. Seek him out. O, untimely death!

A quick perusal of the study I cited suggests that of all the skills in the game, insight may be one of the few that actually is a good model of reality.

That's a curious conclusion.

Does this not show to you how your system is stupid?

Nope, no hostility at all.

Why would I be hostile towards you personally? I have no idea who you are.

That's the fascinating part, isn't it?

So let me get this straight, you don't perceive your responses as being personally hostile AND you're an engineer? Honestly, I couldn't be more shocked.

My hostility is against an idea which
1) Is being created for no reason: mechanics for what you want to achieve already exist.

Not official mechanics. But nevertheless is that a reason for hostility? I'm not creating mechanics in your game....

2) Has been admitted by you to be less than ideal...

Errr...yes. I had an idea that I was looking for feedback on, even though I know it's not perfect and am willing to admit it. And asking for input because I'm willing to consider that it's not a great idea. That's called "inquiry" and "discussion". I'm not trying to win the internet. How about you?

If we made bombs with the reliability of our lie detecting we would never risk going to war. If we could detect lies with the reliability of our electronics manufacturing we wouldn't have crime. (Oh my god, I can already hear the argument about why we would still have crime because blah blah blah blah blah.)

Yet, as the saying goes, you want to take the existence of twilight as evidence that there's no difference between day and night.

The irony in all of this is that you seem to think that you've succeeded at your game theory skill roll, and therefore are 100.0% certain that you are right. And yet...

I'll bow out of this "conversation" with:

Saeviomagy · Jan 23, 2017

Elfcrusher said:
Well, if you can't see the difference I'm not sure I can help you.

Oh, yikes. If you want an RPG that models 6 Sigma I think you'll probably have to write it yourself. (And probably play it yourself.)

My point was that engineering, especially the sort that would be modelled in D&D, is not 100% guaranteed success. Your argument about detecting a lie is that since it's not 100% guaranteed, it's nothing more than intuiting and hunches.

But take a gold star for pedantry, anyway.

I'm an engineer, pedantry is what I do.

Uhhh....yeah. I'm not claiming to be clever or new, was just asking what people thought of it. And upstream I pretty clearly said that an opposed roll can sort of accomplish the same thing as long as it's secret; the only difference is what probabilities result, and how aware the players are of those probabilities. (And also that there are cases (i.e. choosing between passages) where it's not obvious who or what is rolling which opposed skill.)

You specifically said that what you'd come up with was not built into any RPG.

I'm sorry that you don't seem to understand that my system, as flawed as it may be, produces different probabilities.

Right. Could you detail those probabilities to me? I don't think you've actually worked them out.

Slave, thou hast slain me! Villain, take my purse: if ever thou wilt thrive, bury my body, and give the letters which thou find’st about me to Edmund earl of Gloucester. Seek him out. O, untimely death!

Well, despite you saying that this thread is about finding out what people think, and discussing the system, you don't actually seem to be trying to address any of the problems with it, simply to discount the arguments against it.

That's a curious conclusion.

The general population's ability to catch a lie lines up with a DC 10, ~0.25%-0.5% of the population has a +7 mod in it (Which is at least in the same ballpark as the odds of rolling an 18 for wisdom, with some reduction for choosing to increase that stat, plus pick a feat to boost it), and even the people best at it don't have 100% success (which also fits). Other skills have this thing where people who are great at it will still fail on tasks that literally anyone could succeed at.

Nope, no hostility at all.

Your own description was that you're telling the player he should pick the red wire because he had a dream filled with red? But that's still a skill check, and part of the reason you're inventing this rule?

That's the fascinating part, isn't it?

So let me get this straight, you don't perceive your responses as being personally hostile AND you're an engineer? Honestly, I couldn't be more shocked.

Not official mechanics. But nevertheless is that a reason for hostility? I'm not creating mechanics in your game....

Errr...yes. I had an idea that I was looking for feedback on, even though I know it's not perfect and am willing to admit it. And asking for input because I'm willing to consider that it's not a great idea. That's called "inquiry" and "discussion". I'm not trying to win the internet. How about you?

You've continued on with some fairly heavy defense of it's flaws, including poopooing it's weird fortune reversal and making a claim that despite published studies to the contrary, detecting lies is an impossible skill that nobody can master.

If we made bombs with the reliability of our lie detecting we would never risk going to war. If we could detect lies with the reliability of our electronics manufacturing we wouldn't have crime. (Oh my god, I can already hear the argument about why we would still have crime because blah blah blah blah blah.)

If bombs were hand-made one-offs made by a skilled artisan, then you might have a point here. There are an awful lot of incidences of home-made explosive devices that fail to detonate or kill their creators. The bombs we go to war with come off an assembly line after an extensive engineering effort to pin down a reproducable, reliable design, with QA and all the rest of it and yet there's still plenty of examples throughout both world wars of them failing to work as advertised.

Yet, as the saying goes, you want to take the existence of twilight as evidence that there's no difference between day and night.

I'm not sure how a metaphor about slippery slopes applies here.

The irony in all of this is that you seem to think that you've succeeded at your game theory skill roll, and therefore are 100.0% certain that you are right. And yet...

No, just fairly certain that I rolled higher than you.

I'll bow out of this "conversation" with:
View attachment 80373

Oh no! A guy on the internet arguing on a forum has accused me of being on a forum arguing with a guy on the internet!

Lanefan · Jan 23, 2017

Uncertainty, where certainty is not realistic, is good.

For things like trap finding/disarming, secret door searches, and nearly all social interactions uncertainty is very much in play. All a DM has to do is ease off on absolutes "You disarm the trap" and add in qualifiers much more often "You think you disarm the trap". For secret door searches and such-like I always roll in secret as they've no way of knowing whether there's anything there to find at all. I'm not sure there needs to be anything more to it, but on any failed roll a DM should be able at her discretion to give false information e.g. telling someone they find a trap when the roll is '2' and there's in fact no trap there at all...or is there?

As far as things like this causing players to distrust the DM - WTF? It's the DM's job, in part, to keep the players guessing; and if that's enough to cause distrust I can't help you there. And as players we have to accept the very realistic fact that every now and then we/our characters are going to mess up big time no matter how careful we are and that the consequences are gonna hurt...or kill.

Lan-"certainty, like so many other things, is on a non-binary sliding scale"-efan

Gardens & Goblins · Jan 23, 2017

Lanefan said:
As far as things like this causing players to distrust the DM - WTF? It's the DM's job, in part, to keep the players guessing; and if that's enough to cause distrust I can't help you there.

My players know they can trust me to try my best to engage, engineer and support a fair, fun game.

But they rarely, if ever, trust any NPC, door or innocent wildlife I introduce - at least at first. Because they're players and this is D&D

Guest 6801328 · Jan 23, 2017

Lanefan said:
For things like trap finding/disarming, secret door searches, and nearly all social interactions uncertainty is very much in play. All a DM has to do is ease off on absolutes "You disarm the trap" and add in qualifiers much more often "You think you disarm the trap".

If you mean using a secret DC, the issue I have with that is that the player has only the strength of his roll, not the difficulty of the challenge, on which to base his estimate of his probability of success. So, for example, on a low roll the player will probably guess he has failed, but what if the trap was an easy one and he has in fact succeeded? That's a false negative, which is something I'd like to see, but not I think in a good way; narratively I think it takes something away from the game.

Perhaps the DM could create some categories ("easy", "extremely hard", etc.) to share with the player, but at that point we've both added a new element to the roll and kept something secret, which I'm claiming are the two requirements to making this work.

Alternatively we could use the opposed roll. As long as it's kept secret then the player will doubt his chances of success. Again, though, he's not going to know what his chances were in the first place; all he has to go on is his own roll, with no knowledge of what he was facing.

D&D 5E Modeling Uncertainty

Bawylie

A very OK person

Saeviomagy

Adventurer

Guest 6801328

Guest

Guest 6801328

Guest

Saeviomagy

Adventurer

Guest 6801328

Guest

Saeviomagy

Adventurer

Lanefan

Victoria Rules

Gardens & Goblins

First Post

Guest 6801328

Guest

Similar Threads

Voidrunner's Codex