A discussion of metagame concepts in game design

[pemerton]

We can predict that an object I drop tomorrow will fall to the ground. That's not science. The science was all the observation and testing that went into allowing us to predict gravity.

Suppose someone does a series of experiments, dropping various objects various distances, and carefully measures the time they take to fall. Those resuts could be published as tables which might then be useful for various purposes

That would be an example of scientific knowledge - careful measurment use to produce a systematic body of knowledge - which was not about the discovery or explanation of a causal process.

Knowing that it will get cold in winter isn't scientific knowledge, but careful measurement might produce more precise and systematic knowledge, such that - for instance - the likelihood of the temperature in any given day in some particular summer month failing to exceed 20 degrees is such-and-such. That scienitifc knowledge might then be useful for, say, horticulturalists even though it does not identify or explain any causal process that governs temperatures, nor enable the top temperature on any particular summer day to be forecast.

Careful measurement, identifying which measurements can be reliablty extrapolated to future instances, and organisting such measurements systematically so that they are accessible and applicable knowledge - that is one of the things that science does, which distinguishes it from mere common sense observations such as "unsupported objects fall" or "winter is cold".

And to go back to the tangent that spawned this tangent: I don't think that the default gameworlds of fantasy RPGs assume anything about what the results of such careful measurements would be. They don't need to, as it's the nature of most fantasy RPGing that common sense tropes - "unsupported objects fall", "winter is cold", etc - are sufficient to permit players to sensibly declare actions for their PCs, and GMs to adjudicate the results of those action declarations.

 

[Ovinomancer]

The claim "They just got lucky" is not the end of the discussion, but the beginning, and in asserting it [MENTION=23751]Maxperson[/MENTION] was grasping towards the concept of statistical significance.

Don't get me wrong: I'm not a scientist, it's been a long time since I've used this math in any serious way, and it's entirely probable that I will make a mistake along the line here. But I do know the difference between statistical significance and proof.


It seems like you're throwing out the baby with the bathwater here: using a mistake that people make as reason to discard the whole pursuit rather than to say, "oh, we should be careful not to make that mistake".

Actually, I'd be quite happy with the complete demise of the p-value. There are really only a handful of good cases for frequentist stats (RF being one), but you could do with Bayesian techniques (harder, but less misleading).

I think the fundamental disagreement here is just that you have a narrower definition of "science" than I would consider conventional. I'm with [MENTION=42582]pemerton[/MENTION]: if a statistical model built through observation and experimentation allows us to make predictions better than otherwise, then we have learned something in a way I would call "science", even if we don't understand the causation we're capturing in the model yet.

Well, I started with this, so...

Science is the scientific method. Everything else is smearing the term to promote overconfidence in methods and results. Stats isn't science, although it's occasionally useful. Personally, I don't think correlations are anything but sources of new questions to with which science starts. The problem is how often it stops at correlation.

 

[pemerton]

Scientific navigation is another example of correlation without knowledge of causation.

Scientific navigation depends on knowledge about compass needles pointing north; about the motion of the sun in the sky; about the keeping of time by clocks.

I don't think reliable clocks can be built without knowing quite a bit about causal processes within a bit of machinery; but the motion of the sun in the sky, and how that correlates to differences in the time at which noon occurs at different longitudes, can be known without knowing what causes the earth to rotate about its axis at a uniform rate; and one can know that compass needles point north without knowing how magnetism works, or why the earth has a magnetic field - the discovery that the earth's core includes a lot of iron happened much after the use of compasses in navigation had been systematised.

Central to the scientific method is the systematic generation of measurable results, and the ordering of those results so as to enable knowledge - by generalising from them, by determining the conditions under which they permit reliable predictions, etc. Identifying patterns of correlation is a very important part of the scientific method.

An example of the application of scientific method, including the use of statistics, to social rather than natural scientific problems, is demography and related fields (eg public health and epidemiology). This generates systematic knowledge about life expectancies, patterns of morbidity and mortality, etc. It does not have to generate knowledge of causal processes (and there's an argument that it cannot, give that the causal processes - such as transmissions of pathogens from individual to individual - don't operate at the population level).

Of course one can see bad arguments made in public health, because they rest on a conflation of correlation with causation - to put a really crude example, which is perhaps an unfair exaggeration even of the worst arguments, the fact that highly educated people are in the top quartile for life expectancy and low morbidity doesn't mean that if (somehow) everyone achieved those levels of population we could level the population up to those standards (given eg that now we would have people with PhDs working in mines, and thus exposed to risks of workplace injury and death that at present are faced only by those with lower levels of education).

But that doesn't mean that the statistical information that such arguments draw upon is not knowledge, or that the generation of that information is not science.

 

[Shasarak]

Well, I started with this, so...

Science is the scientific method. Everything else is smearing the term to promote overconfidence in methods and results. Stats isn't science, although it's occasionally useful. Personally, I don't think correlations are anything but sources of new questions to with which science starts. The problem is how often it stops at correlation.

I dont really understand your claim that stats is not science. How can you do Science without using stats? If I want to create a new medicine to prevent Heart attacks then how can I prove its effectiveness without the use of stats and of course randomised control studies?

 

[pemerton]

I dont really understand your claim that stats is not science. How can you do Science without using stats? If I want to create a new medicine to prevent Heart attacks then how can I prove its effectiveness without the use of stats and of course randomised control studies?

I take it that, on the conception of science being promoted, it's not science until you have a total model of the biochemistry of heart attacks and of the chemistry of your medicine which allows a demonstration of the precise way in which it will affect that biochemistry.

 

[Shasarak]

I take it that, on the conception of science being promoted, it's not science until you have a total model of the biochemistry of heart attacks and of the chemistry of your medicine which allows a demonstration of the precise way in which it will affect that biochemistry.

So you can not do science until you already know everything about the thing you want to do science on?

Isnt that the opposite of science?

 

[pemerton]

So you can not do science until you already know everything about the thing you want to do science on?

Isnt that the opposite of science?

Don't ask me - I'm not defending it, just trying to explain what I take [MENTION=16814]Ovinomancer[/MENTION] to be saying.

You can see my own suggestion as to how to think about what science is in some of my posts over the past page or two.

 

[Shasarak]

Don't ask me - I'm not defending it, just trying to explain what I take [MENTION=16814]Ovinomancer[/MENTION] to be saying.

I did not get that impression from what he was saying.

Meta-analysis, for example, is an important tool used for analyzing multi studies. So is that 'not science'?

 

[Ovinomancer]

Don't ask me - I'm not defending it, just trying to explain what I take [MENTION=16814]Ovinomancer[/MENTION] to be saying.

You should probably stop doing that.

 

[Ovinomancer]

I dont really understand your claim that stats is not science. How can you do Science without using stats? If I want to create a new medicine to prevent Heart attacks then how can I prove its effectiveness without the use of stats and of course randomised control studies?

It's a method of data analysis, yes, and, as such, can be used to do science. It is not science. Doing stats does not make what you're doing science.

You wouldn't actually have to use stats for that experiment, either. Find a reasonable similar cohort of subjects and have a control group, then compare results. Medicine is another place where bad use of statistics is rampant.