D&D General Lies, Darn Lies, and Statistics: Why DPR Isn't the Stat to Rule them All

[FrogReaver]

Oh boy. It's time to whip it out.

Sure, it works great in practice, but how does it work in theory?

I'd like to understand why you made this comment, honestly. This is what I wrote in the conclusion-

Not alot of time but I wanted to get the concept out quickly. I can elaborate more later. For now, see below.

That's both simple and complicated. The primary problem is that D&D, unlike most sports, doesn't have a large catalog of observed games for statistics. Now that we have twitch, and critical role, and other publicly broadcast games, maybe someone could start compiling that.... but that's neither here nor there. But there is always going to be a difference between "white room" stats and statistics in play. This may change with Beyond and the VTT, but we would need to see it in action.

Even if we did. At most we could say, in critical role campaign X with these other players and characters making these decisions this character output this damage. As you have already suggested, there’s no reason to assume that remains the same with in other critical role campaigns, or when not done by critical role or when the players or characters or decisions change.

In sports that mostly works because teams are usually similar year to year and all play by exactly the same rules, etc, but when there’s a big shakeup everyone wonders how things will pan out.

Other than that, the best way to get useful statistics is to run simulations (Monte Carlo simulations & regression analysis) over and over again with different party compositions and different combats and see the results.

Well no, for Monte Carlo sims to work you have to weight the scenarios you are running correctly. There’s no feasible way to do that. Its a similar problem with white room, except white room spells out its specific set of assumptions. And even if we could there’s also the problem of who gets assigned the damage due to a buff or debuff.

There would necessarily be limits to this based upon even more factors (what monsters, how are the PCs making decisions, accounting for spellcasting, accounting for terrain etc.) but it would provide you with more useful information. IMO.

IMO. All the additional assumptions needed to make a team based Monte Carlo sim give you good information is what makes it garbage in comparison. It’s the very reason white room doesn’t assume party composition or other players character decisions. It’s not because they cannot include those assumptions if they desire, but because trying to make those assumptions generally obscures more than they reveal.

Now, I think that the best way to understand how the various abilities work in terms of play, with different synergies between characters and in different situations and different party compositions should be obvious, at least to me.

By seeing how it actually works in play! If you have a dataset that is large enough, that would be ideal.

Even a large enough dataset isn’t forecastable because it’s not like baseball where we have mostly the same teams facing mostly the same opposition all playing by exactly the same set of rules.

I’m not saying DPR is best metric ever, but it’s just that all the alternatives stink even more. D&D isn’t like baseball for a variety of reasons and those differences make predicting contributions really hard - unless we limit ourselves to a specific set of assumptions and a single character - and while not perfect it’s easy enough to recalculate on the fly for other assumptions as needed.

 

[Snarf Zagyg]

I’m not saying DPR is best metric ever, but it’s just that all the alternatives stink even more. D&D isn’t like baseball for a variety of reasons and those differences make predicting contributions really hard - unless we limit ourselves to a specific set of assumptions and a single character - and while not perfect it’s easy enough to recalculate on the fly for other assumptions as needed.

So, I will point you back to the entire essay I wrote.

Just because it's hard, doesn't mean it can't be done. Soccer, for example, is hard. But it is being done.

The Battier problem I mentioned? Yeah, done.

DPR is used because it's easy and it looks impressive. Saying that something is hard is just an excuse for people not to do it.

 

[FrogReaver]

So, I will point you back to the entire essay I wrote.

Just because it's hard, doesn't mean it can't be done. Soccer, for example, is hard. But it is being done.

The Battier problem I mentioned? Yeah, done.

DPR is used because it's easy and it looks impressive. Saying that something is hard is just an excuse for people not to do it.

Can’t be done for d&d without making so many assumptions the results are worthless.

 

[Snarf Zagyg]

Can’t be done for d&d without making so many assumptions the results are worthless.

..... well, I could do an even deeper dive on this issue, but it you read the OP, and you read my response right there ...

and if this is what you are saying? Yeah, no. We will agree to disagree. I am comfortable with my understanding.

 

[Clint_L]

DPR is not the one stat to rule them all. A lot of its preponderance in discussions probably stems from player overlap with MMOs like World of Warcraft, where it's corollary, DPS, really is the one stat to rule them all, unless you are a tank or a healer.

But it's sometimes very important. Like everything, it depends on context. In our play testing of the UA for 2024, we've found that a significant increase in the DPR of pretty much every martial class and subclass, combined with much greater battlefield control to the same characters (weapons mastery, changes to grappling, specific class changes) has altered combat fairly substantially, making mobs much weaker relative to player characters. And see Snarff's post on power creep for more about the implications of that.

 

[FrogReaver]

..... well, I could do an even deeper dive on this issue, but it you read the OP, and you read my response right there ...

it sounds like you don’t want to really discuss this which is fine, but your OP didn’t address my criticism.

and if this is what you are saying? Yeah, no. We will agree to disagree. I am comfortable with my understanding.

your call.

 

[Evaniel]

Brief synopsis of Snarf's latest batch of posts

And I am here for it.

 

[Snarf Zagyg]

Brief synopsis of Snarf's latest batch of posts

And I am here for it.

As Mama Snarf always told me, "Snarf, you're a star. A real star! Distant and dying. Now, get Mama some of that Wild Turkey and some 'ludes."

 

[Staffan]

Teams games .... they are infinitely more complicated. We can refer to this as the "Battier Issue." In basketball, there was a player, Shane Battier, who didn't have very good statistics when measured by "traditional basketball stats" (points scored, rebounds, assists). But whenever he played, the other players on the court played better. In other words, he was doing the things (defense, setting picks, clearing out the opponent for rebounds) that aren't captured in the statistics for basketball. Battier would make everyone else more successful on the team, but none of the traditional statistics would see his impact.

So... you're saying Battier was a bard?

That aside, I think DPR is best used when comparing individual options, not whole classes. For example, how does 2014 Great Weapon Mastery stack up to +2 to Strength? There, it's a useful tool. It is less useful when comparing e.g. fighters to rogues, because (a) they are doing different things in other pillars, and (b) it depends very much on how each character is built.

 

[Snarf Zagyg]

So... you're saying Battier was a bard?

That aside, I think DPR is best used when comparing individual options, not whole classes. For example, how does 2014 Great Weapon Mastery stack up to +2 to Strength? There, it's a useful tool. It is less useful when comparing e.g. fighters to rogues, because (a) they are doing different things in other pillars, and (b) it depends very much on how each character is built.

I agree!

From the OP, which was undoubtedly written by an erudite and attractive individual of the highest moral character:

None of this is to say that DPR is completely useless. To use an analogy, if someone says, "Hey, a rapier does d8 damage, and a scimitar does d6 damage" then noting that d8 > d6 (in isolation, ignoring weapon properties) is useful information!

And that's what DPR, done well, can be. Useful information. But like so many statistics, over-use, or over-reliance on it without understanding the limits and the issues of it leads to hubris and saying that things "suck" without proper foundation. ....

On the other hand, DPR, like batting average, can continue to be a useful component or tool- but primarily so long as its limitations are acknowledged, and its used to compare two things that are already alike. In other words, if you are choosing between two options for your character in order to maximize damage, then DPR is a great tool! It's value diminishes significantly as you start comparing unlike things.