Air bubble in dice-affecting rolls?

[Dedekind]

A chi-squared test would: it tests whether a particular distribution fits the data.

In general, yes, that is correct.

I could find very little evidence, however, about the impact of a single biased number for high df calculations. For example, construct 200 observations where numbers 1-10 are observed 9 times, numbers 11-19 are observed 10 times, and number 20 is observed 20 times. Eyeball empiricism says that this die is loaded. The Chi-squared test, however, doesn't reject, even at the 10% level for df=19 (Chi-squared test statistic is 11). Even if you increase the bias to getting a natural 20 on 28 out of 200 rolls, it doesn't reject (though 29 does).


The typical Chi-square test with dice is referring to d6s (low df) and that is much less prone to this problem.

 

[starwed]

Eyeball empiricism says that this die is loaded. The Chi-squared test, however, doesn't reject

Why should it? Getting one number twice as often as the others is actually not that unusual in such a small sample size. There's a reason people use statistical tests, and it's because our intuitions on the meaning of data are often wrong.

If it were to happen in play, of course, the fact that it happens to be the 20 appearing so often would be pretty suspicious...

And if you were to set up two models, one with a loaded die and one with a fair die, you'd see that your data does favor the loaded die. But it doesn't outright reject the fair die model until you gather a larger sample size.