3E & 4E Love and Hate Polls - What does it mean?

[haakon1]

If you want to conduct an actually scientific poll, then it might mean something.
. . .
mathematically provable to be inaccurate.

Since you have appointed yourself an expert on scientific polling, please conduct one for us.

Failing that, prove to us, mathematically, why these polls are "provable to be inaccurate".

Put your math where your mouth is . . . either that or admit you don't know any more about it than the rest of us.

 

[TheAuldGrump]

And you base this opinion on what?

No real support - but I suspect that it is rather like primary elections - the base and the core vote, while the general election plays to the middle.

E.N. World contains a core of people who are fairly fanatical about the game, and there is a good deal of polarization within that core, but the fact that we are here at all separates us from the vast majority of players and GMs.

So those most likely to vote (and in one case stuff the ballot box and vote the graveyard) are by the fact that they are posting at all, fairly extreme.

The Auld Grump, why do I have an urge to play 'Gonna Stuff Your Ballot Box' on I-Tunes?

 

[haakon1]

As others have indicated - the statistical mathematics governing polling data do not bend to our wills. What we think is not terribly relevant.

I agree. I think a lot of the objections to the polls, couched in terms of statistical validity are actually about 3 issues:
1) Don't like the topic. This is a "nice" way of telling the people discussing it to shut up. I actually find it far more annoying than being told to shut up. Essentially, as an author of one of the polls, I'm being told I'm mathematically illiterate and/or a liar. So a plain "shut your pie hole" would actually be the nice way to put!

2) Don't like the answers. I don't really get this, since the answers aren't precisely clear. I think it's an "is what it is" situation.

3) Like to spout off about obvious statistical issues, correlation v. causation, etc, because we're a bunch of geeks who like to show we're smart and educated. Even if that's not really relevant to the discussion.

There very language of the questions and allowed answers look to be bias-inducing as well.

I wrote the second poll. I'm not sure if the questions are the same, since I don't pay much attention to the wording of them. But the answers are word-for-word the same, in the same order. If the answers are "bias-inducing", presumably they introduce the same bias to each poll. Which to me would mean it's not really a very interesting bias.

Proper interpretation of data is almost Zen, like using the Force. The numbers tell you things, you don't get to determine what they'll say. You must approach the numbers with as few preconceived notions as possible - and that the numbers must have a meaning we can use is a preconceived notion.

Actually, I fully agree on this!

 

[Crothian]

And you base this opinion on what?

Taking a stats course

 

[haakon1]

No real support - but I suspect that it is rather like primary elections - the base and the core vote, while the general election plays to the middle.

E.N. World contains a core of people who are fairly fanatical about the game, and there is a good deal of polarization within that core, but the fact that we are here at all separates us from the vast majority of players and GMs.

So those most likely to vote (and in one case stuff the ballot box and vote the graveyard) are by the fact that they are posting at all, fairly extreme.

The Auld Grump, why do I have an urge to play 'Gonna Stuff Your Ballot Box' on I-Tunes?

I agree. But I think:
a) We don't have an alternative, better source on gamer opinion.
b) The fanatics are likely trend setters for the rest of the D&Der community. Like the cool kids who made Hush Puppies a fad in Malcolm Gladwell's first book, we create the buzz, we bring together the groups, and we decide the games.

Who keeps the metric system down, who controls the British Crown? We do, we do.

 

[Cadfan]

b) The fanatics are likely trend setters for the rest of the D&Der community. Like the cool kids who made Hush Puppies a fad in Malcolm Gladwell's first book, we create the buzz, we bring together the groups, and we decide the games.

While this is probably true to a certain extent, I can't help but feel that it is probably less true than people online think. Its just got that aura about it that... that even if it wasn't true, we'd still tell ourselves that it was.

 

[haakon1]

Taking a stats course

Trying . . . hard . . . to . . . resist . . . academic pissing contest.

Then we're both aware of self-selection bias.

But this is not a poll on Paizo or WOTC's message board's, or even specific 3e or 4e sections of this site -- this is as neutral grounds as there is. There are two separate polls, each tagged as a poll about its edition, each with the same answer set.

So the self-selection bias is in people answering the poll being more passionate than average -- because they are here, and because they answered a poll.

No one has made an argument for why the self-selection bias should be skewed, other than in a more passionate way (thus under reporting "don't know" and "don't care" answers).

So, unless you've got some other point you're not explaining, the "these polls are all completely worthless" arguments are incorrect.

A better argument is the one that a lot more people answered the 4e than the 3e poll. What exactly that tells us, I'm not sure. Perhaps it means that it would be interesting to compare the absolute numbers for each answer on the two polls, rather than comparing percentages. I haven't checked the math on that but it might be interesting.

Of course, you could just say you don't like the topic, but that's not the same as saying, as you basically are, that the polls are crap and the people who wrote them (me and someone else for 4e) and who compare them are mathematically illiterate and/or liars.

 

[haakon1]

Are those polls the type that show, or that can show, who voted for what? If so, it shouldn't be too hard to pick up ballot stuffing...

I believe it was turned off in both polls. If I recall correctly, I had it turned off in the 3e polls because it was turned off in the 4e poll, so I didn't want it to be any different.

As to why the 4e poller turned it off, I can't say since I'm not him, but I would guess it was one of two reasons:
1) The default is to turn it off, IIRC
2) Secret ballot may seem fairer, especially if you're asking people about possibly confidential feelings. Not every hater wants to admit they're a hater in public!

As to the ballot stuffing complaints, I suppose that's possible, but I don't know if the poll software tries to prevent it. I'd hope our fellow gamers don't try to do that . . .

 

[Celtavian]

re

I doubt a later poll will garner accurate results concerning 4E on EN World. I stopped frequenting EN World shortly after I lost interest in 4E as it has become a 4E centric gaming site since it is bound up with the success of WotC.

I only started coming back once Pathfinder launched, since that forum is now somewhat active.

Any later polls on the EN World site showing differences between 4E and 3E are most likely biased towards 4E because the majority of folks that still frequent this site are folks that have embraced 4E. Most people that prefer 3E have quietly accepted that the new version of D&D is not to their liking and left for other games with other web sites or are quietly continuing to play 3E and sticking to those forums. For all practical purposes, EN world is pretty much a 4E site now with some leftovers from the previous 3E era.

I imagine Pathfinder will have to garner alot of popularity to challenge that. No idea if it will. I'd love to see which gaming system had a more positive effect on the RPG community: 4E or 3E. I'd put my money on 3E.

 

[Crothian]

this is as neutral grounds as there is.

No it is not. This is a battle ground site. It's been so bad the mods have had to ban discussions that would lead to endless arguments. We have trolls from both sides making hit and run posts.

From an academic view the questions and answers are not well written. There is little security on the polls, so I imagine it wouldn't be hard to vote more then once. People can see the results before they vote. There is no analysis or real interpretation of answers just people guessing by looking at raw data.