Open Letter to WotC from Chris Dias

Shemeska said:

Yep, just try paizo and wotc. That's an interesting one there since they're both quite specific.

Click to expand...

or Wizards of the Coast (you know their actual name ) and Paizo

Google Trends: "wizards of the coast", paizo

renau1g said:

or Wizards of the Coast (you know their actual name ) and Paizo

Google Trends: "wizards of the coast", paizo

Click to expand...

Interestingly in the news articles press "wizards of the coast" eclipses Paizo but in web searches Paizo eclipses "wizards of the coast"

renau1g said:

or Wizards of the Coast (you know their actual name ) and Paizo

Google Trends: "wizards of the coast", paizo

Click to expand...

Paizo does have an advantage of also being a word by itself. Look at the results for Greek in the Languages section.

Obryn said:

For starters, because your question is posed to the experimental group that you are targeting. Second, because it is asked consistently of all participants. There's rigor and intent behind the line of questioning; it's not simply a pool of volunteered stories from self-selected individuals.

Click to expand...

But the store owners I asked aren't self-selected. I selected them, and I included in my sample all the store owners I had access to. It's a convenience sample, but a sample nonetheless.

The main difference between anecdotes and surveys is methodology, as I've said. Surveys have methodology behind them, including population selection, bias control, and validity studies. Anecdotes have none of these.

Click to expand...

There is nothing about a collection of anecdotes that precludes those attributes.

And I can't believe I'm explaining this to someone who claims a background in research.

Click to expand...

Yeah, me neither.

And all of these can nevertheless controlled across the population. "Placebo" does not mean, "let your cancer have a field day." When we're talking new cancer treatments, the baseline is someone who's already receiving treatments of some kind - just not the kind you're investigating.

Like I said, this is basic, rudimentary experimental design.

Click to expand...

You're right, it is basic, rudimentary experimental design. So you don't know what a placebo is. Fine.

Is it rude for me to point out that you have now adopted my baseline for cancer research and contradicted your previous post?

I'm not going to speak to anthropology in specific, beyond some basics, because it was not my field of study. (I would ask which sub-discipline you're talking about, though.) I'll happily speak to psychology, which is, and which shares a good deal of overlap.

Click to expand...

Then speak. You happened to pick a field I know little bit about.

In no social science that I'm aware of is anecdotal evidence acceptable. Given that surveys are frequently used in social sciences, I would expect that you'd know the difference between the two.

Click to expand...

You would expect I would know what exactly? I know you'd like to draw a neat little line from anecdote to "only anecdotal" but I don't think you've justified that point. In any case my original point was directed at someone else. Justify why I should argue with you about the criteria for truth.

The essence of logic is to make the best use of the knowledge we have, rather than the knowledge we wish we had. This conversation is a non-starter if you are not willing to admit as evidence that which is most likely. Claiming we need a "controlled study" is fatuous. Controlled studies are a very specific kind of methodology. Outside clinical scienes they are rare, and within clinical sciences, they beg for periodic field studies to confirm their predictions.

Q: What can you say to explain the situation to a D&D 4e 3pp with two black eyes?

A: Nothing, it's obvious they don't listen.

xechnao said:

Interestingly in the news articles press "wizards of the coast" eclipses Paizo but in web searches Paizo eclipses "wizards of the coast"

Click to expand...

Yeah, but isn't "paizo" is a greek term that means "to play"?

Some of those google stats could be from people using the word genericly and not referring to the RPG publisher.

I think WotC already produces more high-quality player and monster options than any group of players could possibly exhaust in a lifetime of play. So I don't think there's any real need there.

I think the only real need is for more adventure and setting content.

Setting content, you don't really need WotC for. Anyone could publish a system-agnostic setting.

Adventure material is the only place I see any real need, where that need has any dependency on WotC. If WotC is smart...it'll use the DDI to allow anyone to make encounters and adventures using their own tools, share them on the DDI with other subscribers who can vote it up or down, or take it and modify it to make their own versions which might be better.

Give the users the tools, let them make the content, give them a robust mechanism for sharing and voting, and we won't need any 3rd party support at all, and we'll get all the high-quality adventure material we could ever need.

pawsplay said:

But the store owners I asked aren't self-selected. I selected them, and I included in my sample all the store owners I had access to. It's a convenience sample, but a sample nonetheless.

Click to expand...

A convenience sample is a very poor sample.

There is nothing about a collection of anecdotes that precludes those attributes.

Click to expand...

The very definition of anecdotal evidence precludes those attributes.

Since you want to link to wikipedia - seriously, check it out.

You're right, it is basic, rudimentary experimental design. So you don't know what a placebo is. Fine.

Is it rude for me to point out that you have now adopted my baseline for cancer research and contradicted your previous post?

Click to expand...

I think you might want to re-read my post, if that's what you think.

A placebo, quite simply, is a sham treatment. It's generally going through the motions - taking a sugar pill, getting a harmless injection, and so on.

Placebos don't preclude the existence of other forms of treatment. A placebo is not, "ha-ha, die of cancer." It's "continue with whatever we're doing, or else start the standard treatment, and we're going to pretend to do something more to you, only you don't know that we're pretending."

With non-medical experiments, where there's no ethical concerns about withholding the best treatment and just ethical concerns regarding deception, it's not even this complicated.

Then speak. You happened to pick a field I know little bit about.

Click to expand...

Simply put - moreso than many other sciences, psychology needs to adhere to methodological rigor. This is in part because it's been historically so full of unscientific fluff, and in part because the mechanics of cognition aren't directly observable. But, the same rules apply as apply in any evidence-based discipline - yes, this includes the social sciences. If you are not applying any rigor to your data, then you have flawed data.

You would expect I would know what exactly? I know you'd like to draw a neat little line from anecdote to "only anecdotal" but I don't think you've justified that point. In any case my original point was directed at someone else. Justify why I should argue with you about the criteria for truth.

Click to expand...

We're not talking about truth per se; I'm not going to get into a metaphysical debate with you, because that would be even further afield than we are. What I am saying is not just that your evidence needs to support your conclusions. Your evidence needs to qualify as such, and you need to show your work. If you're using a collection of stories from game shop owners you know, you know their opinions on the subject and little else.

The essence of logic is to make the best use of the knowledge we have, rather than the knowledge we wish we had. This conversation is a non-starter if you are not willing to admit as evidence that which is most likely. Claiming we need a "controlled study" is fatuous. Controlled studies are a very specific kind of methodology. Outside clinical scienes they are rare, and within clinical sciences, they beg for periodic field studies to confirm their predictions.

Click to expand...

No, I'm saying that you should at least apply some methodological rigor. "Convenience samples" and collections of anecdotes have none.

Anecdotes aren't useless - they're a great starting point for actual research. They're not to be mistaken for research.

-O

BryonD said:

I agree with your observations about game store sales information. But it is still a survey of a group. And we end up declaring one group representative and another not.

Click to expand...

Not specifically. I think it's fair to say that the population of gamers who frequent hobby shops is different from (but likely at least partially overlapping with) the population of gamers who buy from RPG now and that is in turn different from the population of gamers who buy from B&N and that is also different from the population of gamers who shop at Amazon. There's overlap and interplay here, but I have no idea how much.

And when you get games store's data, and distributor's data, and book store data, and 3PP data, and simple personal observations, and so on and so on, and you keep getting the same conclusions for Group A' A'' A''' A'''' etc... it may be very reasonable to not be fully convinced, but it becomes absurd to believe the opposite is true.

It isn't just a matter of saying "game store sales are not representative" of the whole. It is a matter of trying to discredit each and every data piece, one by one, and then declare that because none of them represent the whole, that the collective of them provide no window whatsoever on the whole.

Click to expand...

Well, I think it's reasonable to ask (1) How valid is the data you're looking at? (2) Have you shown that the data you're looking at is evidence for or against your argument? and (3) What pieces of the puzzle are you missing, and might those missing pieces of evidence serve to nullify your tentative conclusions?

In this case, if you're arguing that DTRPG doesn't sell many 3pp 4e products, all available evidence shows you're right. If you're arguing that this means there's no market for 3pp 4e products and that nobody is interested in them, you simply don't have evidence to generalize to this level of abstraction.

-O