Statistics and data can’t answer the types of questions we are asking. There’s no way for the data to differentiate the character from the player. As such the best data can tell us is how much a class contributes with an average player using it. The case could very much be the opposite with expert players or poor players. Then there’s the DM, which matters even for published adventures. Then there’s published adventures which many don’t actually play.
All your data would be able to tell me at the end of the day would be how average players playing a class with a specific build perform in the average dms run through of an average published adventure. Thats not a question anyone cares about.
Recognizing the limitations of data is the most important part of analytics.
It depends on the amount of the data. The assumptions that you are using are likely to be incorrect, given a large enough sample size.
For example, if you are using just the data set of Critical Role, then it will necessarily be skewed because of the difference between that game and other games.
On the other hand, if your data set was the theoretical "all games of 5e, ever," then it would include expert players, poor players, and average players. Arguing against the data set would be similar to the baseball player who says keep complaining about the one time he was "robbed" of a home run in his career, forgetting that over the long term (4-5 at bats per game, 162 games per year, 15 year career) the sample size of "robbed" vs. "lucky carry over the wall" likely balances out.
Or, as I keep saying, there are at least two other good ways to do the math:
A. Use actual data; or
B. Run repeated Monte Carlo simulations and regression analysis.
DPR is a very basic tool, and isn't really "analytics." It's just a more advanced version of saying, "Should I use a dagger (2.5) or a short sword (3.5)?" It's fine to resolve a binary choice, but it's explanatory power doesn't really work past that.
Last edited: