The Pitfalls of D&D Beyond Data

[FrogReaver]

A quick example of the multiclass problem: 2 Characters, A Fighter 10 And a Cleric 1/Wizard 9. Making a class chart as they did would give the following percentages:

Fighter 33%
Cleric 33%
Wizard 33%

Hopefully that makes it apparent just how egregious treating multiclassing like that can be. Not one person is going to breakdown the classes of those 2 characters as shown above... Not one person, but that is how D&D Beyond breaks that down in their class circle graph.

 

[FrogReaver]

A quick example of the subclass problem: Consider having 10 Clerics and 10 Fighters. Suppose 6 clerics and 6 fighters are level 1. 3 Clerics and 3 fighters are level 2. And a single cleric and a single fighter is level 3.

All 10 clerics will have a subclass. For this example the only cleric subclass is life cleric. Only a single fighter will have a subclass. For this example the only fighter subclass is champion.

Thus 91% of subclasses are life clerics and 9% of subclasses are champions despite have the same number of fighters and clerics. The only thing the high cleric subclass percentage shows in my example is that clerics get their subclass at a lower level than fighters get their subclass.

 

[Guest 6801328]

Come on, man. Challenging the methodology by which people draw inferences from a data set is like half of science...

Ah. I didn't realize that's what this thread was, "science":

I want to add. In most business you have the business side and the technical side. I'm sure the technical people get everything we are saying. However, the business side wanted a few clean easy to present charts to show off to the public. They likely hammered away at exactly what they wanted over the objections of the technical people and ended up with what we have.

Nope, no bias there. Just pure scientific inquiry.

My bad.

 

[Mistwell]

The subclass chart was the biggest offender. It's headers didn't accurately define what it was showing. It was showing the breakdown of active characters given that they had a subclass. The circle graph header defined it as showing the breakdown of subclasses of all active characters. So whether or not he declared it accurate it wasn't.

What really happened is you mistakenly inferred something that was never said, and then knocked that strawman down. All the title said was "Subclass Distribution". It never said "all" active characters. It is one reasonable interpretation of that heading to read, "of those with subclasses this is how they were distributed". That's how I read it, it's how some others read it, and it's how the author intended it.

But as if that were not odd in itself, you then learned from the author what it meant, and they after you learned that, you continued debating in this thread without saying a word about that clarification and acted as if the question were still up in the air when you already knew what it meant. You already had been told, "I'm not going to have time to deep dive an explanation on this, but you're not thinking of the data sets how they should be considered." and also, "No, but it is possible to have a higher percentage in a separate data sample that removes characters that have not reached a level appropriate to choose a subclass, which is what has happened here. This data is intended to provide a broad view into relative popularity between all other individual subclasses." And mind you, that was TWO DAYS AGO you found that out. And all this time in this thread you've acted like you didn't have that information!

So yeah, this thread is pretty bizarre at this point.

 

[TheSword]

I'll leave the relevance question open a moment. How does your post address overcoming any of the pitfalls I identified? How does it overcome the issues around multiclassing or subclassing or product costs or "made vs played"? You see, regardless of how well you can classify the population of D&D Beyond and compare that sub-population to the whole D&D population all the pitfalls I mentioned in my OP will still apply. Is there some other pitfall you think your discussion would overcome?

Because it depends entirely on the question you’re asking. The points you’ve raised are not pitfalls, they’re some of the limitations of the data. Your posts suggest people are being hoodwinked or falling into a trap. In fact you haven’t demonstrated that.

That the data doesn’t take into account the differences between free and paid sources isn’t a pitfall in and of itself it’s just a feature of the data. Now if you said the statistics have been inflated because X class isn’t available under free sources or tier 3-4 and therefore has been exaggerated then we could discuss whether other sources like polls, or our personal experience supports that.

For example I can see some subclasses being under represented based on that divide. However we also see Wizard classes being the most popular at high level (despite many higher level spells not being available in free sources) which could suggest the free source ‘pitfall’ is not a factor.

The interesting (and relevant) discussion to be had is to explore your ‘pitfalls’ and to see if they would have an actual impact using specific examples. Otherwise this is all so much hot air.

 

[FrogReaver]

What really happened is you mistakenly inferred something that was never said, and then knocked that strawman down. All the title said was "Subclass Distribution". It never said "all" active characters. It is one reasonable interpretation of that heading to read, "of those with subclasses this is how they were distributed". That's how I read it, it's how some others read it, and it's how the author intended it.

But as if that were not odd in itself, you then learned from the author what it meant, and they after you learned that, you continued debating in this thread without saying a word about that clarification and acted as if the question were still up in the air when you already knew what it meant. You already had been told, "I'm not going to have time to deep dive an explanation on this, but you're not thinking of the data sets how they should be considered." and also, "No, but it is possible to have a higher percentage in a separate data sample that removes characters that have not reached a level appropriate to choose a subclass, which is what has happened here. This data is intended to provide a broad view into relative popularity between all other individual subclasses." And mind you, that was TWO DAYS AGO you found that out. And all this time in this thread you've acted like you didn't have that information!

So yeah, this thread is pretty bizarre at this point.

#1. The subclass chart was titled “subclasses (active characters)”. That designation means it’s a breakdown of the subclasses of all active characters. That isn’t what it showed. Instead it was a breakdown of all active characters with a subclass.

#2. Before he even claimed that’s what that particular chart meant I had already deduced that was what it meant. He didn’t correct my understanding on it, I called out the chart for what it was and he validated my understanding.

 

[FrogReaver]

Because it depends entirely on the question you’re asking. The points you’ve raised are not pitfalls, they’re some of the limitations of the data. Your posts suggest people are being hoodwinked or falling into a trap. In fact you haven’t demonstrated that.

That the data doesn’t take into account the differences between free and paid sources isn’t a pitfall in and of itself it’s just a feature of the data. Now if you said the statistics have been inflated because X class isn’t available under free sources or tier 3-4 and therefore has been exaggerated then we could discuss whether other sources like polls, or our personal experience supports that.

For example I can see some subclasses being under represented based on that divide. However we also see Wizard classes being the most popular at high level (despite many higher level spells not being available in free sources) which could suggest the free source ‘pitfall’ is not a factor.

The interesting (and relevant) discussion to be had is to explore your ‘pitfalls’ and to see if they would have an actual impact using specific examples. Otherwise this is all so much hot air.

Before I get into all this, do you mind to answer my question about what specific thing that I identified as a pitfall that your previous discussion proposal will help us overcome?

I’m fine with the discussion moving on, but since you accused me of dismissing a relevant post I just want to figure out it’s actual relevance. So can you help me identify which thing I identified as a pitfall that the previous post you accused me of dismissing actually addresses?

 

[Laurefindel]

A quick example of the multiclass problem: 2 Characters, A Fighter 10 And a Cleric 1/Wizard 9. Making a class chart as they did would give the following percentages:

Fighter 33%
Cleric 33%
Wizard 33%

Hopefully that makes it apparent just how egregious treating multiclassing like that can be. Not one person is going to breakdown the classes of those 2 characters as shown above... Not one person, but that is how D&D Beyond breaks that down in their class circle graph.

The immediate breakdown that comes to my mind is that, taking your example:

10 levels out of 20 (50%) are fighter
1 level out of 20 (5%) is cleric
9 levels out of 20 (45%) are wizard

You’d need to compare for each character level (i.e. 20 times) so that a 10th level fighter doesn’t “weigh” ten times as much as a 1st level fighter, but that’s not out of scope of what D&D beyond can do.

The science of statistics is not my forte, but it is a well established one. It should be easy(ish) to apply since they possess all the data and shouldn’t have to do much projection past active vs test characters (which even with their method surely imply a certain margin of error). Therefore, I tent to trust the presentation of this data.

If there is any pitfall, it would be to believe that D&D beyond, regardless how accurate the presentation of their data, provides an accurate reprensentation of all gaming circles. I don’t think the D&D beyond statistics are very representative of the tendencies of these forums, but then again, I don’t think the tendencies of these forums are representative of the majority of D&D players.

At any case, none of the data that was presented seemed out of place for me, and I wouldn’t dismiss the popularity of the champion fighter (and life cleric, and berserker barbarian) as skewed because they are easily accessible and available. Those factors must be part of the statistics.

 

[FrogReaver]

The immediate breakdown that comes to my mind is that, taking your example:

10 levels out of 20 (50%) are fighter
1 level out of 20 (5%) is cleric
9 levels out of 20 (45%) are wizard

You’d need to compare for each character level (i.e. 20 times) so that a 10th level fighter doesn’t “weigh” ten times as much as a 1st level fighter, but that’s not out of scope of what D&D beyond can do.

.

This is fantastic. I certainly have no issue with that methodology and I can’t imagine anyone would. I find it amazing and unfortunate that’s not the way they handled multiclassing in the circle graph they gave us.

It solves everything With multiclassing IMO.

 

[Guest 6801328]

I don’t think the D&D beyond statistics are very representative of the tendencies of these forums, but then again, I don’t think the tendencies of these forums are representative of the majority of D&D players.

That could be a disclaimer attached to most of the threads here. Lots of interesting discussions, but too little recognition (in my opinion) that we are the outliers in this hobby. And maybe also too little recognition that our fanaticism, and our tenure*, do not give our opinions and preferences any elevated importance.

*I always laugh when somebody "casually" tries to work into their arguments how long they've been playing D&D, as if that gives them some kind of authority.