Reply to thread

Message

<blockquote data-quote="mamba" data-source="post: 9130561" data-attributes="member: 7034611">Fair enough, neither am I, I will base the following entirely on[URL unfurl=&quot;true&quot;]https://www.pewresearch.org/methods/2018/01/26/how-different-weighting-methods-work/[/URL]so let's see where we agree and disagreeFirst, what this method is intend to do is to reduce the bias in a self-selecting poll (like WotC's). The bias it tries to adjust for is introduced by the people answering the poll not being an exact proportional match for the entire population, and different groups within the population holding different opinions to varying degrees.Second, it attempts this by determining the deviation of the people answering the poll from the overall population and then adjusting the poll results towards what is known about the overall population (from well established factors that are known for the entire population, e.g. from government surveys), based on demographic factors such as sex, age, race and ethnicity, educational attainment, and geographic region.Feel free to disagree and say where I misunderstand this.Based on this, I'd say it cannot be applied by WotC. For one they do not ask for enough demographic factors for this to be feasible, and second there are no established facts that are known for the entire population (by such demographic factors) that would be of any use here.The paper then goes on to discuss that this approach is not really working reliably in the first place (&quot;But are they sufficient for reducing selection bias in online opt-in surveys? Two studies that compared weighted and unweighted estimates from online opt-in samples found that in many instances, demographic weighting only minimally reduced bias, and in some cases actually made bias worse.&quot;), but that is moot here, as this is not a workable approach for WotC's survey to begin with, for the reasons I gave above.</blockquote>

[QUOTE="mamba, post: 9130561, member: 7034611"] Fair enough, neither am I, I will base the following entirely on [URL unfurl="true"]https://www.pewresearch.org/methods/2018/01/26/how-different-weighting-methods-work/[/URL] so let's see where we agree and disagree First, what this method is intend to do is to reduce the bias in a self-selecting poll (like WotC's). The bias it tries to adjust for is introduced by the people answering the poll not being an exact proportional match for the entire population, and different groups within the population holding different opinions to varying degrees. Second, it attempts this by determining the deviation of the people answering the poll from the overall population and then adjusting the poll results towards what is known about the overall population (from well established factors that are known for the entire population, e.g. from government surveys), based on demographic factors such as sex, age, race and ethnicity, educational attainment, and geographic region. Feel free to disagree and say where I misunderstand this. Based on this, I'd say it cannot be applied by WotC. For one they do not ask for enough demographic factors for this to be feasible, and second there are no established facts that are known for the entire population (by such demographic factors) that would be of any use here. The paper then goes on to discuss that this approach is not really working reliably in the first place ("But are they sufficient for reducing selection bias in online opt-in surveys? Two studies that compared weighted and unweighted estimates from online opt-in samples found that in many instances, demographic weighting only minimally reduced bias, and in some cases actually made bias worse."), but that is moot here, as this is not a workable approach for WotC's survey to begin with, for the reasons I gave above. [/QUOTE]

Verification