Reply to thread

Message: <blockquote data-quote="tetrasodium" data-source="post: 7948079" data-attributes="member: 93670">That came up in a recent thread & I answered it <a href="https://www.enworld.org/threads/is-5e-the-least-challenging-edition-of-d-d.670422/post-7935154" target="_blank">here</a>. along with some examples  The way you ask a question influences the responses you get.  Pew research has an excellent article that goes into those sort of things <a href="https://www.pewresearch.org/methods/u-s-survey-research/questionnaire-design/" target="_blank">here</a>.  Without getting into politics there is another example of it that you might be familiar with if you read/watch the news regularly you will frequently hear how a poll says voters are in hate/love soandso's proposal but when those same voters are polled on what they feel about specific policies based on the policy rather than name the have the opposite opinion on those specifics.  The same also applies to the order you ask the questions inOn the topic of magic items & using some mentioned already, some examples might be:<ol> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "I sometimes have trouble tracking all of my equipment"</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "The wand of magic missile is easy to use"</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree, " I think the wand of magic missile & others like it would be a lot of fun (chosen simply because it was posted earlier).</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "I think the wand of magic missile would fit well in the party"</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "The wand of magic missile is well balanced"</li> </ol>Not only do those start out with questions that prime the mental pump to push a responder into the player's headspace, they all biased in ways that are bordering on push polls.  compare to something like:<ol> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "players sometimes have excessive difficulty  tracking details of their equipment such as asf acp crit range crit threat brutal# damage reduction etc "</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "The wand of magic missile daily recharge mechanic rather than a set number of charges is a good design choice"</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree, " I think the wand of magic missile & others that recharge like it would be an appropriate treasure for a level 4 party to find.</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "I think the wand of magic missile would fit well in the party"</li> <li data-xf-list-type="ol">On a scale of 1-5 with 1 being strongly disagree & 5 being strongly agree.  "The wand of magic missile reharge mechanic is well balanced"</li> </ol>Right off the bat, the difference between the first question 1 & second question 1 is that the first only asks if the responder "sometimes" has difficulty tracking things.  Alex could vote 5 because at high levels he has six pages of inventory that has nothing to do with the complex bits missing from 5e equipment. Beth could vote 5 because she likes to fit 5 pages of stuff on two and does so by writing in <a href="http://printwiki.org/Mice_Type" target="_blank">mice type</a>.  Charles could vote 5 because he tries to avoid filling out new character sheets & eventually starts erasing through the paper. Dawn could vote 5 because she has dyslexia.  None of those four voted five because the items themselves are too complex. .... The second question 1 specifically points out what potentially difficult to track aspect about the equipment is the subject of being difficult to track so Alex looks at his six pages & decides those particular aspects are not a significant hurdle enough to merit a 5. Beth looks at her mice type & decides that those numbers are not the problematic thing that her mice type was for the first question 1.  Charles looks at those things and admits hose values almost never change on a weapon/armor so are not the cause of his erasing through the page problem so again does not vote 5. Dawn thinks about those values & admits that they are not her dyslexia so also does not vote 5The first question 2 is so simplistic  & vague that it is effectively meaningless.  An autohit spell you just roll some d4's with is obviously easy.  Uttering the words "I use my wand of magic missile to cast magic missile at $target" is beyond the point of merely being "easy".  The thing gets back all it's charges within a couple days with d6+1 (avg 5, min 2!)/day & has nearly zero chance of ever going away, what could be easier?  everyone votes 5 on a question that should never have been asked to an individual capable of taking the survey without assistance for reasons other than language or physical disabilities on par with those of stephen hawking before death.... but hey it got a bunch of 5's so must be good!The second question 2 asks about the charge mechanic specifically  By using the words "good choice" rather than easy to use or fun, responders are forced to consider the broader implications of going from set number of charges to what is effectively making the item permanent and more specifically do so in comparison to the old charge mechanic.   Alex thinks back to that time he burned out a few wands he'd been saving in his back pocket as his ace in the hole to save the day & realizes he would have just had one or maybe two wands that couldn't have enough charges to repeat that under this system so does not vote 5.  Beth Realizes that her five pages of inventory will never happen because hose items are now both more powerful & permanent that she won't have a stockpile made from odds & ends collected  & she's always enjoyed being the group's record keeper so certainly does not vote 5. If charles really thinks keeping track of the inventory not tracked by Beth is a significant problem he might vote higher than  Alex & Beth, but he might also have other reasons for not doing so.  Dawn might look at that mechanic & think about how it would complicate her experience due to dyscalculia so tries to vote less than 1 but can only vote 1.The First question 3.  Would it be "fun"?... sure  why not, Wotc seems to really like asking this kind of question in the UA surveys.The Second question 3. Alex looks at that and realizes "oh my god, I thought that was on par with staff of fireball or something not an uncommon low level magic item!" he does not vote 5 because he realizes that such a wand would be a significant part of what defines his character.  Beth comes to similar realizations & thinks about how the shift to very few character defining magic items would impact the feeling of fulfillment she gets being the group's record keeper so also does not vote 5. Charles thinks about how he's going to be not only subtracting but also adding charges to the wand regularly so avoids 5 because it means he's going to be erasing through the page more often.  Dawn comes to whatever xonclusion she comes toBoth Question 4s are the same, Would it fit well in the party?... all four think & say "I guess?..." to themselves but because of question 1 & 2 the second version is likely to have lower results among the same four if asked the second set insteadThe first question 5.Is it balanced?  what part?  the spell magic missile?  The max charges?  The 1d6+1 recharge/day?  The upcasting costs?the attine requirement? The uncommon rarity?  what a uselessly vague question, wotc likes these kind of things in UA surveysThe second question 5.  Alex thinks back to the earlier questions he was asked from the second set & his answer is likely to look a lot like his answer to 2 & 3 but could be a little higher or much lower depending on if he tries to stay neutral & objective or if he weights it down against what he' be losing.  so on & so forth.With that said, many of those questions I raised in regards to those four players & the questions are examples of some of the things  a well designed survey should attempt to determine with additional questions & the UA surveys almost never seem to.  In this context, one reason those things are important is that it allows people to be grouped based on their preferences in order to determine if there are additional things important to those people that should be included or represented in new ways rather than accidentally making a well meaning change catering to a specific group that damages the experience for a second group.  A second reason might be to weight those groups because both Andy & Alice are outnumbered 4:1(at least)  by their players, but if those players rarely if ever GM or think that being a gm is too hard for them Alice & Andy's oppinions need to carry much more weight in some cases.  Those extra questiona also allow decisions to be made so that design choices can include support for the kind of games both andy & Alice rather than just one.  That last point is especially important if the designers are all mostly a single type of GM demograpic like andy likely to not even consider or be aware of the needs that a different style of game like Alice's  in their design (see the class feature UA video <a href="https://youtu.be/v7A6-yOZC_s?t=420" target="_blank">from earlier</a> of crawford saying almost exactly that).</blockquote>

Verification