Gamesurvey.org – academic research on hobby gaming community

bayonetbrant

Explorer
Without looking at the list again, I can't be more specific, but the list is EXTREMELY board and computer focused... for example, RPGs have almost no 4X games, but do have romances, and 2X and 3X games (which 2 of the 4 X's vary; D&D tends to be explore and exterminate.)

We actually stripped out a LOT of the digital/computer-focused items from '06. But it's trying to cover the entire swath of hobby gaming all at once, and isn't going to be overly-focused on any one specific type of game. It's funny, b/c the card guys don't think its granular enough for them. The wargamers don't think it's granular enough for them ("why aren't block games their own separate option?")
No way we can dive that deep for everyone :)
 

log in or register to remove this ad


bayonetbrant

Explorer
Why is the survey so broad then? You have a facile response for every problem we point out. That is not how you learn to do better.

(1) Survey is so broad b/c we're interested in the entire hobby games world rather than multiple separate-but-overlapping efforts that inevitably take more time and are more difficult to join for a holistic view of the overall hobby.
Any time you end up releasing a limited-focus hobby gams survey, what inevitably happens is that enthusiastic people will share it beyond the audience on which you're focusing, and you get a combination of extraneous responses and frustrated respondents. An example is the annual WSS survey that always gets shared beyond the miniatures wargaming crowd, which results in the board wargamers, fantasy skirmish minis gamers, and others, providing input of limited utility and complaining that it doesn't address their specific concerns.
We've tried to broaden ours to cover as many possible hobby gamers as we can, and there are plenty of gamers that overlap many different kinds of games. While I've got over 500 different RPG products on my shelves, I've also got several hundred wargames, over 100 Eurogames, multiple boxes of minis of all kinds. At this point in my life I'm primarily a wargamer, but I've been playing RPGs since '81.

(2) We're going to take all this feedback into future efforts and use it to improve later research. What we can't do is make mid-stream changes to something that's already got over 1300 responses, b/c that'll potentially invalidate what's already collected. We learned a lot from 2006 and used that feedback for this one. We'll use the feedback from this one to improve the next one (and I hope it won't be a 14-year gap again!). The issues raised above by the person with vision challenges is definitely useful and something to make sure we consider next time.

(3) As to responses for every issue you guys point out - I love the feedback b/c it helps us improve and shows that people care enough to get involved. So that's great! But if you guys are asking questions that we've already encountered or discussed or dealt with in previous survey research, should I not respond? I'm making a good faith effort to engage so I can help explain our thinking behind what was done. Where we've screwed up, I think we've owned that (I'm not a fan of the Qualtrics UI, but I also don't have the option to have FDU change their survey software contract).
But there are other times where people bring up issues that were intentional (like the inverting of scales). Based on the comments, we might well go into a future study and decide "precedent be damned, we're changing this based on previous feedback". For now, though, there's a reason behind what we did. Would you prefer I not explain it?
 

MNblockhead

A Title Much Cooler Than Anything on the Old Site
I'm partially sighted. I was doing the survey on a landscape-format laptop screen, with the font sizes turned up far enough that I have about six question rows on the screen at a time. With the screen like that, it's much, much easier to keep track of which end of a scale is positive mentally than keep scrolling back up to check.
Even with surveys where the scale is consistent, I with more platforms would have the lines under the header on likert-scale tables scroll with the header staying fixed.
 

MNblockhead

A Title Much Cooler Than Anything on the Old Site
(1) Survey is so broad b/c we're interested in the entire hobby games world rather than multiple separate-but-overlapping efforts that inevitably take more time and are more difficult to join for a holistic view of the overall hobby.
Any time you end up releasing a limited-focus hobby gams survey, what inevitably happens is that enthusiastic people will share it beyond the audience on which you're focusing, and you get a combination of extraneous responses and frustrated respondents. An example is the annual WSS survey that always gets shared beyond the miniatures wargaming crowd, which results in the board wargamers, fantasy skirmish minis gamers, and others, providing input of limited utility and complaining that it doesn't address their specific concerns.
We've tried to broaden ours to cover as many possible hobby gamers as we can, and there are plenty of gamers that overlap many different kinds of games. While I've got over 500 different RPG products on my shelves, I've also got several hundred wargames, over 100 Eurogames, multiple boxes of minis of all kinds. At this point in my life I'm primarily a wargamer, but I've been playing RPGs since '81.

(2) We're going to take all this feedback into future efforts and use it to improve later research. What we can't do is make mid-stream changes to something that's already got over 1300 responses, b/c that'll potentially invalidate what's already collected. We learned a lot from 2006 and used that feedback for this one. We'll use the feedback from this one to improve the next one (and I hope it won't be a 14-year gap again!). The issues raised above by the person with vision challenges is definitely useful and something to make sure we consider next time.

(3) As to responses for every issue you guys point out - I love the feedback b/c it helps us improve and shows that people care enough to get involved. So that's great! But if you guys are asking questions that we've already encountered or discussed or dealt with in previous survey research, should I not respond? I'm making a good faith effort to engage so I can help explain our thinking behind what was done. Where we've screwed up, I think we've owned that (I'm not a fan of the Qualtrics UI, but I also don't have the option to have FDU change their survey software contract).
But there are other times where people bring up issues that were intentional (like the inverting of scales). Based on the comments, we might well go into a future study and decide "precedent be damned, we're changing this based on previous feedback". For now, though, there's a reason behind what we did. Would you prefer I not explain it?
I, for one, appreciate your engagement. I hope you'll post the results and links to any published papers here later.
 

bayonetbrant

Explorer
Even with surveys where the scale is consistent, I with more platforms would have the lines under the header on likert-scale tables scroll with the header staying fixed.

I'm not 100% sure of the limitations of the Qualtrics software. Even if it doesn't give you a fixed header where the items scroll below it, we should be able to break the questions up by repeating the header more frequently.

I think this one is imminently fixable for next time, for sure.
 

bayonetbrant

Explorer
I, for one, appreciate your engagement. I hope you'll post the results and links to any published papers here later.

Thanks, and absolutely. We fully intend to share what's published far & wide.

By way of expectations: we'll have some overviews published at the Dragoons site fairly quickly at the end of the Summer. Anything submitted (and, we hope, accepted) for academic publication will definitely take longer.
 

aramis erak

Legend
We actually stripped out a LOT of the digital/computer-focused items from '06. But it's trying to cover the entire swath of hobby gaming all at once, and isn't going to be overly-focused on any one specific type of game. It's funny, b/c the card guys don't think its granular enough for them. The wargamers don't think it's granular enough for them ("why aren't block games their own separate option?")
No way we can dive that deep for everyone :)
You'd probably have done better to have separated the CCG/LCG/ExpandableCardGame, board games, TTRPGs, LARPS, and Computer Games into 5 separate categories, since many people are into multiple of those.
Or at least to explicitly include all the categories up front.
It would definitely have changed my answers on pages 2 and 3 to have realized it wasn't just the tabletop games... because I answered solely on TTBG and TTRPG... Tho the game I want to get more table time is an LCG model.
 

bayonetbrant

Explorer
Would you like to know more?

Join Liz of Beyond Solitaire as she interviews Brant about the project, and answers questions from the audience, too

1630 EDT on Monday 5/6

 

Remove ads

Top