Reply to thread

Message

<blockquote data-quote="tarchon" data-source="post: 1060085" data-attributes="member: 5990">The first big problem I see a lot when people discuss random number generation is this term &quot;fair&quot; - what does it mean to you?&nbsp;&nbsp; Equal probability of outcomes, uncorrelated outcomes?&nbsp; The other big problem is the lack of specification for accuracy and confidence for the test.Asking &quot;is this die fair?&quot; is like asking &quot;is my car too green?&quot;&nbsp; More information is needed to answer the question usefully.Testing the mean gives you some information, and it's information you can get fairly accurately with a comparatively small number of trials.&nbsp; Whether it's a useful statistic depends on what you're doing with the numbers generated.&nbsp; With MdN stat generation, that's an excellent parameter to look at, since it is efficient to measure and you probably don't care much if 4 comes up a little more frequently than 5, as long as it's balanced out somewhere else.&nbsp; With thresholding mechanics, like d20 to hit, what's more important is the likelihood of each outcome, or more specifically, the equalities of the outcomes.&nbsp;&nbsp; That (usually) takes more trials to get accurately, however the test of the mean is also sensitive to defects in the frequencies of various outcomes.&nbsp; Whether it's sensitive enough depends on what accuracy you require and what kinds of &quot;unfairness&quot; you need to detect, which practically also depends on what kinds of &quot;unfairness&quot; are likely to occur as well as how significant they are.&nbsp; It makes little sense to design a test based on its ability to determine whether the die only rolls 1s and 20s - does that happen a lot?&nbsp;&nbsp; If you simply test outcome frequencies, that could also fail to detect a die that always rolls numbers in pairs, like 20, 20, 8, 8, 17,17..., however this mode of &quot;unfairness&quot; is similarly so unlikely that it's not worth designing a test around it.&nbsp; Now, people who gripe about computer PRNGs, to bring in a related FAQ, are griping about such a lack of independence between trials, which to me is usually so small in degree as to be irrelevant for game dice rolling, but that can be tested too.Anything anybody wants to know about die outcome probability distributions can be tested with standard formulas to a known degree of accuracy and confidence, but you first have to know how to describe what you want to test.</blockquote>

[QUOTE="tarchon, post: 1060085, member: 5990"] The first big problem I see a lot when people discuss random number generation is this term "fair" - what does it mean to you? Equal probability of outcomes, uncorrelated outcomes? The other big problem is the lack of specification for accuracy and confidence for the test. Asking "is this die fair?" is like asking "is my car too green?" More information is needed to answer the question usefully. Testing the mean gives you some information, and it's information you can get fairly accurately with a comparatively small number of trials. Whether it's a useful statistic depends on what you're doing with the numbers generated. With MdN stat generation, that's an excellent parameter to look at, since it is efficient to measure and you probably don't care much if 4 comes up a little more frequently than 5, as long as it's balanced out somewhere else. With thresholding mechanics, like d20 to hit, what's more important is the likelihood of each outcome, or more specifically, the equalities of the outcomes. That (usually) takes more trials to get accurately, however the test of the mean is also sensitive to defects in the frequencies of various outcomes. Whether it's sensitive enough depends on what accuracy you require and what kinds of "unfairness" you need to detect, which practically also depends on what kinds of "unfairness" are likely to occur as well as how significant they are. It makes little sense to design a test based on its ability to determine whether the die only rolls 1s and 20s - does that happen a lot? If you simply test outcome frequencies, that could also fail to detect a die that always rolls numbers in pairs, like 20, 20, 8, 8, 17,17..., however this mode of "unfairness" is similarly so unlikely that it's not worth designing a test around it. Now, people who gripe about computer PRNGs, to bring in a related FAQ, are griping about such a lack of independence between trials, which to me is usually so small in degree as to be irrelevant for game dice rolling, but that can be tested too. Anything anybody wants to know about die outcome probability distributions can be tested with standard formulas to a known degree of accuracy and confidence, but you first have to know how to describe what you want to test. [/QUOTE]

Verification