A discussion of metagame concepts in game design

[Maxperson]

As well as the examples you point to (astronomy) we could point to aspects of biology (Darwin didn't do experiments - he made very careful observations and conjectured the best explanation for them given the constraints he took to be applicable) or demography/public health (no one goes about spreading diseases or polluting water supplies to try and determine the effect on life expectancy, birth rates, and other aspects of population health), etc.

His observations and measurements were the experiment in this case. He would compare the same species on different islands(an experiment) and note the differences in their adaptions.

 

[Ovinomancer]

That's the grade-school presentation of the scientific method. In real sciences of many varieties, "experiments" have to take the form of making further careful observations of the world and assessing whether they conform to a prediction made by the hypothesis. For example, controlled and repeated experiments are not usually possible in astronomy: if we want to test, say, the prediction made by general relativity about gravitational lensing, we have to wait until a solar eclipse and then measure the apparent positions of the stars around it. Make no mistake: these "natural experiments" are not as good as controlled experiments in a laboratory. There is greater risk of the result being a consequence of an uncontrolled variable, and as far as repeatability goes we're at nature's mercy. We have to be more careful, and cannot be as certain. But that does not mean that no science can come of it. It seems absurd to declare that the Eddington-Dyson observation of the 1919 eclipse, one of the most dramatic and famous experimental confirmations of a hypothesis in the history of science, was "not scientific".

Yes, you can conduct experiment by observing phenomena. You must have a theory and observe to falsify, and, as you note, such experiments have less power. Still, there are things you can only observe, but thise are also often areas where we have limited knowledge.

The common problem is to gather a data set, do analysis, discover a correlation, and deckare this as science. It is not. You must form a theory and test it against newly gather data. Then you've done science. There's a large set of current study that skips the experiment and relies on stats on the original collection. Not science.

A further issue is the creation of computer models and testing the model. This can be a useful tool to narrow down where to look for more data, but it is not science. Model output is not valid data. Most egregious in this area are doing stats on stats, like many if the fMRI studies. fMRI is a statistical model by itself; doing statistical comparisions of statistical models is turtles, all the way down.
[MENTION=42582]pemerton[/MENTION]. Collection and cataloguing of data cannot be science by itself, else baseball scorekeepers are engaged in science. Collection of data is necessary to science, but it is not sufficient.

 

[Ovinomancer]

But, as [MENTION=92239]Kobold Boots[/MENTION] put it, you're drifting away from the mainstream there.

That's tautological. What's the scientific method? Why doesn't observation-based statistical prediction qualify?

Is the mainstream always right, or somesuch? What a strange argument.

And, yes, it appears tautologucal, yet it seems extremely controversial, yeah? And there are a few arguing for science being things outside the method or only one part of the method. If you're (general you) going to argue science is a means to truth, then you should stick to the proven method. This is really the summation of my argument.

 

[Ovinomancer]

[MENTION=16814]Ovinomancer[/MENTION], re: reification. I have pondered your posts, and the point is well made. I had never before encountered the term in the context in which you used it. As it happens to be one of my favorite words, I felt it necessary to do some digging. I am no statistician, but I don’t need to be; I fully understand the fallacy of treating a model of a thing as the thing itself. But this line is more interesting:


Because 400 years ago, astronomy was that literally. Like alchemy, it needed to shake off the woo-woo, and enter respectability.

My sense is that with economics, sociology, psychology, we are observing phases in nascent sciences; the disciplines are still struggling to cohere. Psychology is hardening – largely because of neuroscience – but it still has a long way to go. The traditional physical sciences are more established, and have had longer to iron out their wrinkles – perhaps that is why the soft edges of the soft sciences are frustrating to those who are grounded in the hard.

I would submit that history is one of the youngest sciences; historians seem singularly resistant to any kind of systematic inquiry and display an almost paranoid avoidance of mathematics. Bayesian reasoning is beginning to appear at the fringes; it is not being well-received – because history professors are ignorant and lazy. My evidence is purely anecdotal.

And what I think matters not one whit. If a discipline really is a science, it will self–correct and improve, and demonstrate its value – such is the virtue of science; if not, it goes the way of homeopathy, confined to a few fringe cranks who seek ever-deeper meanings in its arcane formulations.

I disagree history could ever be a branch of science -- there's no way to experiment. It could certainly do with more rigor.

 

[Ovinomancer]

His observations and measurements were the experiment in this case. He would compare the same species on different islands(an experiment) and note the differences in their adaptions.

Darwin's theory is only the first step of science. He saw something, had a question, did research, and firmed a tgeory. We're still collectively doing the experiment/falsification steps. It's ongoing.

I rather like the theory of evolution and think its most likely right. We've certainly found some surprises that have required modifying the theory so far, but nothing that's falsified it.

 

[Kobold Boots]

On this I tend to go with Orwell in his essay on Newspeak.

More words allows nuance, rhythm, assonance, alliteration, etc. It increases the expressive power of the language.

Reading on, I see that [MENTION=4303]Sepulchrave II[/MENTION] has said the same in reply.

Also, on statistics and causation: scientific knowledge isn't limited to knowledge of causal processes. Statistically confirmed correlations may enable predictions to be made, even though the causal process that generates the correlation is not known. This is starting to push my knowledge of the history of science, but I would have thought that Mendelian genetics and 19th century statistical mechanics of gasses would be examples of scientific knowledge of correlations in ignorance of the actual causal process.

Fair. However, I think it's more clearly stated by others I've met, that correlation does not indicate causation.

 

[VisanidethDM]

This thread could work perfectly as the origin story for a Frenzied Berserker.

 

[TheCosmicKid]

This is a controlled experiment. The control isn't as great as the lab, but it's there.

And controls on an election, or on a living human body, or on the weather are also not great, but to some extent there. This isn't a binary "experiment"/"not experiment" distinction. It's a continuum from "better control" to "worse control".

We can know with incredible accuracy when solar eclipses will happen. I watched one earlier this year or maybe it was last year. I can't remember. I was able to go outside exactly when it happened because of repeatability of experiments in the past.

You're putting the cart before the horse. The timing of solar eclipses is itself a scientific prediction, and going outside to observe it at the predicted time is an experimental confirmation. But, like you say, the controls on this experiment aren't perfect. Maybe aliens read our news and blot out the sun when eclipses are predicted as a prank. We can't control for that.

Also: not to be flippant, but we can know with incredible accuracy when presidential election will happen too.

Whereas you cannot repeat an election. Even going to a second term, so much has changed that Trump's second attempt at election cannot possibly be viewed as a repeated experiment. There is no science involved in these sorts of predictions. You might as well get another Paul the Octopus to do it.

Well, that's a hypothesis, and we can test it. We can continue watching elections and seeing whether in the long run, Nate Silver or Paul the Octopus makes better predictions. Yes, variables change. It is precisely because variables change that we repeat experiments. In principle, with a perfectly controlled experiment, we wouldn't have to.

 

[TheCosmicKid]

Yes, you can conduct experiment by observing phenomena. You must have a theory and observe to falsify, and, as you note, such experiments have less power. Still, there are things you can only observe, but thise are also often areas where we have limited knowledge.

The common problem is to gather a data set, do analysis, discover a correlation, and deckare this as science. It is not. You must form a theory and test it against newly gather data. Then you've done science. There's a large set of current study that skips the experiment and relies on stats on the original collection. Not science.

Okay, now we're getting somewhere. I agree with all that. My point is that modeling an election and then testing it against an upcoming real election certainly seems to count as the former rather than the latter. Nate Silver isn't looking at past data and saying, "JFK won the 1960 election! Hah, called it!"

 

[TheCosmicKid]

I disagree history could ever be a branch of science -- there's no way to experiment. It could certainly do with more rigor.

There are a couple of ways to experiment in history. You can do what we've been talking about: make a prediction about some phenomenon and then wait for more history to happen and see if the prediction bears out (e.g., "based on the trajectories of totalitarian states, the North Korean regime will fall within 25 years"). You can also make a prediction and test it against new data you gather in the primary sources and archeology (e.g., "this culture suffered a plague at this date, so we should see an increase in death motifs in their writing and art").

The problem isn't the inability to experiment. It's the tiny sample size - just one timeline, no repeats - and the fact that the experiments are way down on the "worse control" side of the continuum. It's certainly never gonna be a hard science.