The Pitfalls of D&D Beyond Data

[Mistwell]

As to value, I think their actual dataset is very valuable even with questionable representativeness. However, The graphs we got to see are mostly invaluable but there’s still a sliver of value to them once we are able to label what they are actually showing vs the incorrect claim of what they are showing.

Invaluable does not mean what you think it means. At least, I assume that, unless you intended to contradict yourself in the same sentence.

Assuming that is the case, what do you feel is "very valuable" about the dataset? Let's see if we can find room for agreement.

 

[FrogReaver]

Invaluable does not mean what you think it means. At least, I assume that, unless you intended to contradict yourself in the same sentence.

Pedantic much?

 

[Zardnaar]

The guy doing to videos and presented the graphs is also the guy who headed the creation of the service, and helped program it. If there is a significant division at DDB between the "business side" and "technical side", he isn't representative of that divide.




There was a person who used to post here, who in an interaction via PM that got us both a deserved vacation, tried to tell me that the forum in general clearly liked him more than me, because he had more XP than me.

Of course, I didn't bother pointing out that he also had been here for twice as long, and had twice as many posts, but only a couple hundred more XP...I just don't care about internet popularity contests enough to win that easily won "contest", I guess.

People are weird, is the point.



Is that why you come here? Certainly isn't why I'm here.



Also see, 5e rangers. Most people aren't satisfied with them, according to wotc polling, but they are in the high middle of classes played, according to wotc AND DDB data. Because people play concepts that they like, even when the mechanics aren't good.



Active characters were loosely defined in the most recent dev update. I'm not going to go watch it a second time just to answer more precisely, but it was something along the lines of "characters who get updated". So, ya know, active characters.

I don't think the Hunter Ranger is that bad, power wise its even good and fighters are a bit meh IMHO until higher level (Battlemaster being an exception). Kind of figured out the forum hivemind was wrong back in 3.5 where Fighters were rolled up plenty, Druids were very rare (saw 1 or 2), and I saw very few wizards as well (more bards believe it or not).

The charisma based classes and races seem popular in 5E though.

 

[Azzy]

Pedantic much?

He IS a lawyer.

 

[FrogReaver]

He IS a lawyer.

That explains EVERYTHING!

 

[Mistwell]

Pedantic much?

Ironic that you focused on this and not the thesis of my post ("what do you find "very valuable" about the dataset?"). Pedantic much indeed.

I gave you an opportunity to explain in case you did mean Invaluable. If it was a typo, cool. I make typos all the time and was not trying to pick on you. But, you could have meant something that was not coming across so I thought it fair to ask.

And again, what do you find "very valuable" about the dataset?

 

[Mistwell]

He IS a lawyer.

 

[FrogReaver]

Any time we get data of any sort which is objective hard data, the same thing happens.

If the data doesn't meet people's personal experience or expectations, they make an argument for why the data isn't perfect and therefore should be dismissed.

Of course, data being imperfect does not mean data should be dismissed. Data can be extraordinarily accurate and very fairly representative, while still being imperfect. Statistics has demonstrated, conclusively, that data can be fairly representative of a whole while remaining imperfect.

We should be able to talk about broad generalizations and trends without constantly being reminded that the data isn't 100% perfect and is not measuring all conceivable games and players and DMs and purchasers and experiences in the world.

When you start posting in this thread with accusatory posts like the one above.

And flat out dismissive posts like the one directly below

I was not talking to you.

Nobody has done that.

Prove me wrong.

And argumentative posts like the one above

And another dismissive posts like the post directly below

That's just a title of an article though. Show me people (you said people, as in plural) were saying it. They are not.

And more flat out dismissive posts like the 2 directly below

There is no "the" dispute. I am talking to Frog specifically, and you are oddly answering for him, and in a way I don't think he agrees with by the way though I am sure he will appreciate your general support. I think he does have a quibble with the overall data set and intent. Maybe not, but I feel very confident he did not tag you in to respond for him.

But bottom line here, he questioned the value of the data, I stated that, he quibbled with that though he again repeated he thinks it's inaccurate which is a direct questioning of its value, and I think it's a good demonstration of the kind of thinking were dealing with here when someone dispute using the word "value" while calling it "inaccurate". It's the kind of thing someone who is looking to "win a fight on the internet" would say rather than have a conversation where there is back and forth allowed.

I mean seriously, if I cannot characterize "it's inaccurate data" as "you're questioning the value of the data" then this isn't a conversation, right?

You can jump it of course. But, you can't answer for someone else. Which is what you did. As for you having an issue with my tone, take that to a moderator and not to me.

Anyways, after all THAT (and more), why do you think I care one bit about engaging you on something that is only tangentially related the topic of this thread (like what value can be had from the D&D Beyond dataset that none of us will ever actually be able to see)? Hint: I don't care about your tangent and at this point any goodwill I may have otherwise had toward discussing that particular tangent has been used up in my documented dealings with you above.

I mean seriously... it's almost like you've had it out for me specifically since you started posting in this thread. So I'll ask nicely, if this is a vendetta you have out for me, please drop it.

 

[TheSword]

Any historian will tell you that any source has to be analyzed. However instead of talking about what data doesn’t tell us, isn’t it more constructive to discuss what it does tell us? We can recognize that the data has limits and imperfections but that doesn’t mean it can’t be used to infer information about a broader picture. If I read the diary of a factory worker in 1860’s England, it doesn’t tell me how all workers lived but it might give suggestions for other useful information, items available at the time, current events, forms of entertainment, foods available etc.

Let’s question the data more positively...

1/ Who is more likely to use D&D Beyond? Younger/older/new players/multi edition players.

2./ Who is unlikely to use D&D Beyond?

3./ What are the likely differences between these groups? Home campaigns vs published campaigns.

For instance while the statement “90% of games end before level 10” might be inaccurate. However if we suppose that d&d beyond is more likely to be used by a more IT savvy generation that have come new to the hobby when D&D beyond was sold as a key tool for the game, the phrase might be modified to say...

“90% of games played by the younger, newer players using d&d beyond end before level 10.” This information is still useful to me and I suspect to WOC too. Particularly as we know that younger, new members are responsible for the explosion in sales and therefore people playing d&d.

We can ask ourselves are people using d&d beyond likely to favour different character types to those that don’t. If the answer to this is no, then we are free to assume that this sample is more representative.

We can recognize that it might hide some information like the difference between paid and free rules. However if that’s an issue I would much prefer to discuss what difference we would expect that to make. Rather than try to invalidate the source at face value. For those who say the data is worthless I say you are using it to answer the wrong questions.

 

[FrogReaver]

Any historian will tell you that any source has to be analyzed. However instead of talking about what data doesn’t tell us, isn’t it more constructive to discuss what it does tell us. We can recognize that the data has limits but imperfections but that doesn’t mean it can’t be used to infer information about a broader picture. If I read the diary of a factory worker in 1860’s England, it doesn’t tell me how all workers lived but it might give suggestions for other useful information, items available at the time, current events, forms of entertainment, foods available etc.

Let’s question the data more positively...

1/ Who is more likely to use D&D Beyond? Younger/older/new players/multi edition players.

2./ Who is unlikely to use D&D Beyond?

3./ What are the likely differences between these groups? Home campaigns vs published campaigns.

For instance while the statement “90% of games end before level 10” might be inaccurate. If we believe that d&d is more likely to be used by a more IT savvy generation that have come new to the hobby when D&D beyond was sold as a key tool for the game, the phrase might be modified to say...

“90% of games played by the younger, newer players using d&d beyond end before level 10.” This information is still useful to me and I suspect to WOC too. Particularly as we know that younger, new members are responsible for the explosion in sales and therefore people playing d&d.

We can ask ourselves are people using d&d beyond likely to favour different character types to those that don’t. If the answer to this is no, then we are free to assume that this sample is more representative.

We can recognize that it might hide some information like the difference between paid and free rules. However if that’s an issue I would much prefer to discuss what difference we would expect that to make. Rather than try to invalidate the source at face value. For those who say the data is worthless I say you are using it to answer the wrong questions.

IMO That sounds like a great start to a brand new thread.