The Pitfalls of D&D Beyond Data

[MNblockhead]

That could be a disclaimer attached to most of the threads here. Lots of interesting discussions, but too little recognition (in my opinion) that we are the outliers in this hobby. And maybe also too little recognition that our fanaticism, and our tenure*, do not give our opinions and preferences any elevated importance.

*I always laugh when somebody "casually" tries to work into their arguments how long they've been playing D&D, as if that gives them some kind of authority.

Depends what the topic of discussion is. If it is about, say, how the game was played in the 70s and 80s, I think whether or not you were playing in the 70s or 80s gives your anecdotes more weight than someone who was not.

I also think that someone who has played multiple editions and many systems over many years *may* have more insight regarding, say, the effect of some specific rule on game play.

But if I want to know more about what makes a game attractive to young, new players the grognards may not be the best people to listen to.

In many of my posts I state that I started playing in the early 80s. Stopped in the early 90s. Started again with 5e. My purpose in doing so is simply to give those that I'm engaging in a discussing with an idea of where I'm coming from. It may increase the weight or decrease the weight of any assertions I make.

 

[FrogReaver]

You could just read the post. In essence I said look at the data to decide where it is representative, where it isn’t. What questions can it be used to answer and where do we need to find supporting sources.

the pitfalls Im discussing weren’t really about what the dataset represented but pitfalls that would apply to the dataset regardless of what it represented.

So how is discussing what the data represents actually relevant to that? That’s what I’m trying to understand. If it’s not relevant to that then how is it actually relevant to this thread? And if it’s not actually relevant to this thread why am I being accused of dismissing the post?

I’m open to being wrong and that maybe my viewpoint is a bit narrow at the moment about what is relevant but for goodness sake explain it.

 

[Guest 6801328]

I also think that someone who has played multiple editions and many systems over many years *may* have more insight regarding, say, the effect of some specific rule on game play.

They might, but they could just as easily be misled by nostalgia, or even just "that's what I'm used to", and it's hard to tell the difference. And where genuine insight does exist it should be demonstrated, not merely defended with seniority.

In a disagreement about a game being played today, can you imagine somebody changing their mind just because the other poster was besties with EGG? I can't.

 

[FrogReaver]

Hmm...really? It's possible you misread my intent...I edited for clarity.

I still see nothing there I don’t 100% agree with.

That's totally human. Just not very scientific.

Sure. But what’s the need to be scientific about that specific insight. It wasn’t like the comment that was said in Response to was about a scientific issue in the first place.

I guess ultimately what I’m saying is that every statement in a discussion about math and science doesn’t have to be math and science, and especially not when the statement is a response to a non-math and science comment.

So really what is it that the quote proves? That when I’m not responding about a science and math issue that I’m not scientific but instead offer a reasonable point of view. Why are you making that an issue at all? It doesn’t seem very reasonable to do so IMO.

I don't mean to defend the data or it's conclusions...I'll let @BadEye, seemingly the only person with actual information, do that. But this thread is rife with pre-existing belief that those conclusions are wrong, and arguments that are only intended to support that belief, not to engage in genuine inquiry.

So to wrap it in a shroud of "back off...I'm just doing science" is comical.

I’d love to have this discussion.

Have you stopped to ask us what informs the belief that those graphs are inaccurate and misleading? Or maybe I should take this approach. What informs your belief that those graphs are accurate and not misleading?

I can give you my evidence for that claim. Can you give me evidence for yours?

 

[TwoSix]

I don't mean to defend the data or it's conclusions...I'll let @BadEye, seemingly the only person with actual information, do that. But this thread is rife with pre-existing belief that those conclusions are wrong, and arguments that are only intended to support that belief, not to engage in genuine inquiry.

So to wrap it in a shroud of "back off...I'm just doing science" is comical.

And see, this is where you're flat wrong.

I have no disagreement with the general results that were displayed in the graph. I certainly have no idea of what I would WANT the data to look like. Like, absolutely no concern whatsoever. (i mean, if it said Purple Dragon Knight was the most popular subclass, I'd look at the data a little funny, but besides that. )

I objected to the methodology of ONE graph out of the group that were shown. It's barely an issue at all. But it's still a flaw in what was being communicated in the graph versus the data that was actually gathered. That's all.

 

[MNblockhead]

They might, but they could just as easily be misled by nostalgia, or even just "that's what I'm used to", and it's hard to tell the difference. And where genuine insight does exist it should be demonstrated, not merely defended with seniority.

In a disagreement about a game being played today, can you imagine somebody changing their mind just because the other poster was besties with EGG? I can't.

Oh, I am almost certainly misled by nostalgia and what I'm used to. But I don't think you are arguing that we stick to formal logic in these discussions. There is a lot of subjectivity involved and that is part of what makes these discussions interesting. Experiencing different points of view.

When I share some of my background, I'm doing it in this spirit. Knowing my background may actually help you understand my blind spots and prejudices. Whether somebody's experiences have any persuasive value depends on the context. I certainly don't think one should wield their decades of gaming experience as a cudgel to silence the "newbs" but that doesn't mean experience has no value.

As for the weight of ones relationship with the late EGG on a disagreement about a game, that depends on the nature of the disagreement. Yes, appeal to authority, especially when cut with hearsay, is weak sauce. But it may be the remoulade that brings the dish together.

 

[Guest 6801328]

So really what is it that the quote proves? That when I’m not responding about a science and math issue that I’m not scientific but instead offer a reasonable point of view. Why are you making that an issue at all? It doesn’t seem very reasonable to do so IMO.

It doesn't "prove" anything, except maybe that you're not being completely objective. Rather, I offer it as evidence (again, as distinguished from proof) that you're bringing some pretty heavy biases to the table.

To wit: this thread is ostensibly about what might be wrong with the methodology. In the OP you asked a bunch of perfectly valid questions, but in the absence of answers that invalidate the methodology (or possibly in the presence of answers, from @BadEye which validate it), you jumped straight to inventing explanations for the imagined bad science by assigning blame (to the "business people") based on pure prejudice.

All of which...to get back to my snarky comment on the 1st page...makes me doubt the objectivity, and even the purpose, of this whole endeavor.

 

[Guest 6801328]

Oh, I am almost certainly misled by nostalgia and what I'm used to. But I don't think you are arguing that we stick to formal logic in these discussions. There is a lot of subjectivity involved and that is part of what makes these discussions interesting. Experiencing different points of view.

When I share some of my background, I'm doing it in this spirit. Knowing my background may actually help you understand my blind spots and prejudices. Whether somebody's experiences have any persuasive value depends on the context. I certainly don't think one should wield their decades of gaming experience as a cudgel to silence the "newbs" but that doesn't mean experience has no value.

So far so good. No disagreement. There are lots of perfectly valid reasons to cite one's experience. Nevertheless...

As for the weight of ones relationship with the late EGG on a disagreement about a game, that depends on the nature of the disagreement. Yes, appeal to authority, especially when cut with hearsay, is weak sauce.

I certainly haven't done a formal study, but my impression...after seeing such references countless times...is that it's often (even usually) an attempt to pull rank.

But it may be the remoulade that brings the dish together.

XP for that.

Anyway, this is a total digression. I wasn't trying to hijack the thread.

 

[FrogReaver]

It doesn't "prove" anything, except maybe that you're not being completely objective. Rather, I offer it as evidence (again, as distinguished from proof) that you're bringing some pretty heavy biases to the table.

Given my position that the graph was inaccurate and misleading and the context of someone dismissively asking whether the D&D Beyond staff was incompetent what do you think would have been a thoughtful and unbiased answer to that question?

To wit: this thread is ostensibly about what might be wrong with the methodology. In the OP you asked a bunch of perfectly valid questions, but in the absence of answers that invalidate the methodology (or possibly in the presence of answers, from @BadEye which validate it), you jumped straight to inventing explanations for the imagined bad science by assigning blame (to the "business people") based on pure prejudice.

my position is that I have evidence for my belief that those graphs are inaccurate and misleading so please don’t call refer to it as me “inventing explanations to imagined bad science” at least until you consider my evidence for calling it “bad” in the first place.

So again, given my position of inaccurate and misleading and my evidence of inaccurate and misleading and a question asked about incompetence who do you think it was reasonable for me to blame for the mistake?

All of which...to get back to my snarky comment on the 1st page...makes me doubt the objectivity, and even the purpose, of this whole endeavor.

As most here I match perceived snark with snark. You will find plenty of snark in my posts. Most all of it is justified with the “snark for snark” motto IMO

 

[Guest 6801328]

So again, given my position of inaccurate and misleading and my evidence of inaccurate and misleading and a question asked about incompetence who do you think it was reasonable for me to blame for the mistake?

Huh? I don't care.

I'm just saying that the manner in which you assigned blame is revealing, and contradicts the assertion that this thread is somehow "scientific". Certainly not more scientific than the work it's criticizing.