Reply to thread

Message: <blockquote data-quote="tetrasodium" data-source="post: 8250577" data-attributes="member: 93670">Adding to what [USER=6780330]@Parmandur[/USER] put forward about sampling, the wording of a poll question & order they are asked in can also dramatically influence the results <a href="https://www.pewresearch.org/fact-tank/2018/03/21/how-do-you-write-survey-questions-that-accurately-measure-public-opinion/" target="_blank">for example</a> (<a href="https://www.pewresearch.org/fact-tank/2019/01/29/good-jobs-vs-jobs-survey-experiments-can-measure-the-effects-of-question-wording-and-more/" target="_blank">continued</a>).  Your poll had a single question asking about dozens of things that all assume an improbably high level of system knowledge in every recipient then you tried to use those results to show multiple things.  Even the sampling is badly flawed, I answered the poll here & then ignored it when I saw it on reddit because iirc you asked people not to respond to it in different places, doing that however makes the differences between communities of dubious value at best since there is going to be significant overlap between communities and you've taken steps to excise that overlap.  Even the way you group the respondents is very important & reddit vrs enworld vrs whatever is simply not a meaningful distinction to playing or running d&d.  For example....<ul> <li data-xf-list-type="ul">Are the results of a reasonably designed series of survey questions meaningfully different when asked to players vrs GMs</li> <li data-xf-list-type="ul">Are the results of a reasonably designed series of survey questions meaningfully different when asked to VTT gms/VTT players</li> <li data-xf-list-type="ul">Are the results of a reasonably designed series of survey questions meaningfully different when asked to different regions</li> <li data-xf-list-type="ul">Are the results of a reasonably designed series of survey questions meaningfully different when asked to different regions ages</li> </ul>All of these are things that can be asked with one or two questions each & possibly a survey split for GM/player or VTT/No VTT use to potentially show meaningful differences.I bolded"The polls Wizards does" because Wotc is almost hilariously bad at polling in the vast majority of their polls. Take this common question "Is X fun" yes/no & how fun is X 1-5.  Lets use warlock invocations for X<ul> <li data-xf-list-type="ul">Majority say yes it's fun ->awesome power level for the class is perfect lets make more equally powerful invocations to meet their needs</li> <li data-xf-list-type="ul">Majority says it's not fun ->Hmm that's a shame, clearly it needs to be stronger lets make more equally powerful invocations to meet their needs as they advance to gain more invocations</li> <li data-xf-list-type="ul">Majority rank on the low end of 1-5 -> See bob, I told you that the invocations need to be stronger, I made a bunch of new versions that crank these weak threes up to 4 or 5, look at this repelling agonizing blast & think ho cool it will be</li> <li data-xf-list-type="ul">Majority rank on the high end of 1-5 -> See bob, I told you we were on the right track.  I made this eldritch lance invocation to bump the range on the cool invocations so it's going to be more regularly in range</li> </ul>A question like that doesn't tell you anything about why the respondent answered the way they answered or even how/if the question is relevant to them... are invocations fun?.. <ul> <li data-xf-list-type="ul">"I'm the gm....  uhh.. i guess no cause they are kinda OP.. or maybe yes  cause my warlock player seems happy to be dialed in around 11.  Maybe there are other questions for that"</li> <li data-xf-list-type="ul">"well I felt like the warlock kind of overshadowed the rest of us & noticed a couple players talking about rolling one up.  Plus I'm kinda sick of hearing 'lets take a short rest' at the end of every fight so I guess no... well the warlock player seemed to be having fun so maybe yes cause I guess none of those problems are covered by this & figure there must be other questions for that"</li> <li data-xf-list-type="ul">"I'm the warlock pc & felt pretty badass so guess that's a yes... I didn't really like the frustrated looks always asking us to take short rests got  so maybe no... I heard a couple other players talking about rolling up warlocks & feel like it might cause too many warlocks & that wouldn't be fun... but there must be other questions for that.."</li> <li data-xf-list-type="ul">Surprise there are no other questions for that</li> </ul>In a lot of cases it would be difficult to deliberately make a worse poll than the ones wotc puts out if you tried.  I can only think f a couple exceptions & think both of them were this year</blockquote>

Verification