Reply to thread

Message: <blockquote data-quote="Jeremy Ackerman-Yost" data-source="post: 5279598" data-attributes="member: 4720">JohnSnow, there's no need to specifically emphasize places where I've wandered off into opinion land.  I think it's pretty clear in the text.  <img src="https://cdn.jsdelivr.net/joypixels/assets/8.0/png/unicode/64/1f642.png" class="smilie smilie--emoji" loading="lazy" width="64" height="64" alt=":)" title="Smile    :)"  data-smilie="1"data-shortname=":)" />  Heck, "feel" was the operative word in the part you bolded.  I'm not trying to pass off my experiences for data.  But, until I see some data, I'm entitled to base my interim conclusions on my experiences.   But I'm a scientist by inclination even more than training, so I'm happy to revise conclusions when evidence is presented.Sure.  I was using "survey" in a sort of handwave-y sense encompassing various kinds of self-report and interview.  But something that is common to all of them, including the observational/self-report market research methodology you describe, is a high degree of variability.  Not only are you contending with human behavior, human self-perception, and the wildly divergent characteristics of your subject population, but also the human vagaries of your researchers.  You put 5 researchers in the same room and tell them to take notes on someone's behavior, and you'll get a bunch of notes that look like they are describing somewhere between 3 and 5 completely separate events in disparate rooms.  Now, you can control that by giving them very specific directives and goals to build up your inter-rater reliability, but the more of that you do, the more you bias what they are going to see.If they're all looking for the same things, they will tend to see the same things.  And they will also tend to ignore the same things.  Experimental design can always bias results, but it's an especially common problem in any study of human behavior, because our assumptions are so ingrained.When you construct a methodology for these kinds of studies, you have to make a great many assumptions.  If you're not very careful, those assumptions bias the result.  Heck, I've seen assumptions bias the results in experiments on slime mold behavior, and there's not quite as much variability and noise there as you get with humans, nor as much risk of confirmation bias, IME.On that last point, we are entirely in agreement.  I've simply been exposed to different trends on how "the way their brain works" tracks to demographics.  Age, previous PnP experience, previous non-PnP gaming experience, and other factors all influence these things more strongly, IME, than less nuts and bolts notions like creativity or tactical experience.  Those things are factors, sure, but not as strong for my money.Also, on the topic of outside the box, one of the best pieces of advice I ever saw for 4e came from these boards.  I think it was Piratecat, but don't quote me on that.  He gave his players a power card (encounter, I think?) that said "Do something cool."  If you have that sitting there looking at you while you play, I find that even the very tactical, power-oriented players use it now and again, as long as the DM isn't an ass about it.</blockquote>

Verification