A discussion of metagame concepts in game design

[pemerton]

I wish that was true and on the other hand there are just so many examples of this I dont know which to point you at first.

What have you got in mind? Are you thinking of exposures/infections of soldiers and prison inmates?

 

[Shasarak]

What have you got in mind? Are you thinking of exposures/infections of soldiers and prison inmates?

Ha, I wish soldiers and prison inmates.

How about the Tuskegee syphilis experiment run by the US Public Health service or the Fernald Science Club experiments run by MIT and Kellogg.

 

[Ovinomancer]

Why not? Because it's trivial? Nate Silver himself is first and foremost a sports statistician - national elections are a sideline.

Sorry, I'm uncertain -- are you claiming sports betting is also science? If so, we have little more to discuss on this line, as your definition of science is so broad as to be meaningless.

I'm having a hard time reconciling "knowledge of the game sufficient to make accurate predictions reliably" with "not science".

Yes, I see that. Walk this back a moment -- where is knowledge 'sufficient'? At what point does it switch to uneducated guess? If you can't point to that line (fuzzy or not), then your operational definition of science becomes 'makes a guess'. If science is to have meaning, it has to me more than makes a guess, educated or not.

This is exactly why my definition of science is the method. If it's not the method, it's not science. And you cannot do science on statistics (because it's not data from the real world).

If I'm trying to find the gravitational constant G, I might estimate it at some value, run an experiment, compare the results to what's predicted by my estimate, and then adjust the estimate up or down accordingly and repeat. I am effectively "massaging" the gravitational force in my model. But I'm massaging it in accordance with experimental results, in order to make better predictions.

No, because G isn't the data you're obtaining in your experiment. If you, instead of measuring the experiment, take a poll of the people that were present and use their opinions of the outcome as your data, you're in the ballpark of what I'm talking about. If you then discount Bob because he's blind and favor Paul because he's been pretty okay lately (your opinion) and use that as the data for G, you aren't finding G anymore, you're finding people's opinions on G. And you've massaged your data.

If Nate Silver decides a priori, "My gut tells me Ohio is going to go red", and then weights his data to say that Ohio is going to go red, then you're right, that's not science. But if he does what he says he does, and what we have no reason to suspect he's lying about, then the weights he puts on his data are in response to the results of previous experiments, and will be adjusted in response to future ones, in order to zero in on some value which can make predictions most accurately. That looks like a scientific process to me.

No, I don't think he does that, but he does observe the polls and note which have been successful in the past and weight those more heavily. He tries to "correct" for things he thinks are deficient in the polling executions (bias, sample, date, etc.) and weights the data he has accordingly. But that's subjective as well. It's subject to expertise -- Silver clearly isn't stupid and is actually very good at what he does. But, it's still subjectively altered data as an input. He's using stats on his opinions.

I'm going to take a stab in the dark and guess that you don't read many horoscopes? The key element of horoscopes and prophecies, the trick that makes them work, is that they make vague predictions which can can be interpreted after the fact as conforming to whatever happened, and are thus difficult to falsify. Whatever else you may say about political prognosticators, they do not do that. We can say, definitively, what Nate Silver was right about and what he was wrong about. That clarity is in falsification conditions the sine quae non of science. So to compare a prediction which has such clarity with one which doesn't is deeply unfair and, when the whole point of this discussion is "what is science?", actively misleading.

Are you unaware of error bars? Silver gave central guesses, but his model quite often encompassed both results inside the error margin. Again, his calls on individual states were his opinion on the model outcome, because the actual outcome of his statistical model on his choices of data had error bars that weren't as precise. You can choose to ignore that because his headline guess was often right (he's had a few bombs, as well), but the model you're claiming is the science is not at all as confident. This is available for looking at (one other thing I credit Silver for, he's discloses pretty well).

I don't dislike Silver. I think he is a good pundit. He's got some great observations. It's not science.

With apologies to Francis Fukuyama, history did not end in the year 1990. History is the diachronic study of human behavior and societies - or, if I may be tautological in turn, it's the study of the stuff that makes it into history books. Historians have tons to say about current and future events. They differ from, say, sociologists or political scientists in that their methodology is more focused on finding patterns of cause and effect in the written and archeological record, although naturally the boundary between the disciplines is fuzzy and there's a lot of overlap. But in short, "Those who do not study the past are doomed to repeat it" really is kind of the historian's mission statement.

Just like when a scientist opines on philosophy they're doing philosophy and not science, when a historian opines on current events, he's not doing history. I have a lot to say on music, but that doesn't make me a musician (no truer benefit to the world).

 

[Shasarak]

This is exactly why my definition of science is the method. If it's not the method, it's not science. And you cannot do science on statistics (because it's not data from the real world).

I must admit that I am baffled how data from the real world can not be data from the real world?

Must be an engineering thing, I guess.

 

[Ovinomancer]

Collection and cataloguing of data can be science, by itself, if other conditions are satisfied.

Huh? Those things can be science by themselves so long as other things happen, too? I'm not sure what work you think 'by itself' is doing in this sentence, but it's not any work I'm familiar with!

Joseph Banks systematicallly catalogued fauna and flora on his voyages. That was - in my view, correctly - treated as a contribution to science.

Not science, but very useful to science. What theory did Banks test with his collection? The method is the science, pieces of the method are not.

Astronomy begins by systematically recording the position and visible motion of heavenly bodies. That is science.

Not science, but vary useful to science. Astronomers do science, but it's by observing (research) and forming a question which they try to find support or falsify by further observation. The act of observation is, by itself, not science. Necessary, but not sufficient.

There are many sorts of information that can be collected and systematised. There are many ways of doing so. And there are many degrees of difficulty in doing so. Whether or not such collection and systematisation constitutes science may depend on all these things. Baseball scores are (i) not interesting features of the natural world, (ii) not hard to discover, and (iii) are not systematised in any interesting or revelatory way by a scorekeeper.

Whereas the work done by Banks, or by early observers of the heavens, (i) concerned interesting features of thenatural world, (ii) required the application both of discipline and intellectual effort to discover, and (iii) was systematised in various interesting and revelatory ways.

Your i, ii, and iii are not things that are requirements of science. There's tons of uninteresting science. There's tons of obvious science. There's tons of non-revelatory science. You seem to be making an argument that profundity is a requirement of science. It is not. The method is the only requirement of science.

For example, I could run experiments on the number of rice grains that fall from the tumbler I got in Disney World five years ago if I fill it to the brim with dried, white rice and tip it at a 45 degree angle. That's not (i) an interesting feature of the natural world, nor (ii) hard to discover, nor (iii) systematised in various interesting and revelatory ways, but I can still Science! the hell outta it.

I must say, I am constantly amazed at the strange woo people keep hanging on science. Science must mean something is really profound, or that my favorite result using stats is science, or... I don't get it. The need. The need to have science be important, to be special. It's a fantastic tool, but it's just a tool. It can be used for boring crap or spectacular, world-altering discover. It can be benign and terrible. But, at the end of the day, it's just a tool. And one wielded by people -- and scientists are just people, with all the usual gamut of vice and virtue. They're not special, either, they're just people. I don't get the reverence.

 

[Ovinomancer]

I must admit that I am baffled how data from the real world can not be data from the real world?

Must be an engineering thing, I guess.

Statistics aren't data from the real world. This is why I keep using reification to describe the effect. When you do stats, you start with real data - well, you hope you have real data like measurable observations and not a social science 'instrument'. Then you pick your statistical test, and here's where the hanky-panky occurs. When you pick the test, you pick a set of parameters -- proxy assumptions about your data that now stand in for part of your real data. Then you run the math and get a result, but the result isn't on your data, it's on the parameterization of your data. You make some assumptions and then hide them behind the math. And, viola! You have a math-y result that looks great and convinces you that the model you just built is the real world and that the result isn't hugely overconfident. If your experiment starts with statistics, you've done a bad. There's some areas where you can get away with it, but you should be extremely careful to not misrepresent the results to yourself, much less others.

 

[pemerton]

Huh? Those things can be science by themselves so long as other things happen, too? I'm not sure what work you think 'by itself' is doing in this sentence, but it's not any work I'm familiar with!

It's doing the same sort of work as "per se" or "ipso facto".

Eg hitting a ball with a bat is not per se sport, but it can be if certain other conditions are satisifed.

If the other conditions are internal to the hitting of the ball with the bat, then we're talking about some instances of hittings, diestinguished by their internal properties and relations.

The features I mentioned as making careful observation and measurement science are not external conditions. They are internal to the observation and measurement.

For example, I could run experiments on the number of rice grains that fall from the tumbler I got in Disney World five years ago if I fill it to the brim with dried, white rice and tip it at a 45 degree angle. That's not (i) an interesting feature of the natural world, nor (ii) hard to discover, nor (iii) systematised in various interesting and revelatory ways, but I can still Science! the hell outta it.

Without more, I'm also not persuaded that it would be science.

Science connotes not only a method, but also an output: a contribution to the stock of human knowledge, which is systematically generated and hence (in some fashion) systematically recoverable and disseminable.

As you describe it, I'm also not sure what your "experiments" would consist in other than careful observation and measurment. What properties of the cup, or of the grains, are you suggesting you would be investigating?

EDIT: I also checked the Wikipedia page on Joseph Banks, to see if it confirmed my own recollections:

[A]n English naturalist, botanist and patron of the natural sciences.

Banks made his name on the 1766 natural history expedition to Newfoundland and Labrador. He took part in Captain James Cook's first great voyage (1768–1771), visiting Brazil, Tahiti, and, after 6 months in New Zealand, Australia, returning to immediate fame. He held the position of President of the Royal Society for over 41 years. He advised King George III on the Royal Botanic Gardens, Kew, and by sending botanists around the world to collect plants, he made Kew the world's leading botanical gardens. . . .

From his mother's home in Chelsea he kept up his interest in science by attending the Chelsea Physic Garden of the Worshipful Society of Apothecaries and the British Museum, where he met Daniel Solander. He began to make friends among the scientific men of his day and to correspond with Carl Linnaeus, whom he came to know through Solander. As Banks's influence increased, he became an adviser to King George III and urged the monarch to support voyages of discovery to new lands, hoping to indulge his own interest in botany. . . .

In 1766 Banks was elected to the Royal Society, and in the same year, at 23, he went with Phipps aboard the frigate HMS Niger to Newfoundland and Labrador with a view to studying their natural history. He made his name by publishing the first Linnean descriptions of the plants and animals of Newfoundland and Labrador. . . .

Banks was appointed to a joint Royal Navy/Royal Society scientific expedition to the south Pacific Ocean on HMS Endeavour, 1768–1771. . . .

The voyage went to Brazil, where Banks made the first scientific description of a now common garden plant, bougainvillea . . .

While they were in Australia, Banks, Daniel Solander and the Finnish botanist Dr. Herman Spöring Jr. made the first major collection of Australian flora, describing many species new to science. . . .

Banks arrived back in England on 12 July 1771 and immediately became famous. He intended to go with Cook on his second voyage, which began on 13 May 1772, but difficulties arose about Banks' scientific requirements on board Cook's new ship, Resolution. The Admiralty regarded Banks' demands as unacceptable and without prior warning withdrew his permission to sail.​

It seems that at least a number of scientists of his day regarded Banks's careful observation and measurement as constituting a contribution to scientific knowledge. And I personally regard the idea that one of the more significant enlightenment scientists, who was famous for his scientific endeavours and was a long-serving president of the Royal Society, was in fact not a scientist generating scientific knowledge, as a counterexample to the mooted definition of science and scientist that produce that result.

 

[TheCosmicKid]

Sorry, I'm uncertain -- are you claiming sports betting is also science?

It can be, if the method used to make predictions is the scientific one. Why not? Like you say a little later, science is a tool that can be used for boring crap.

Yes, I see that. Walk this back a moment -- where is knowledge 'sufficient'?

When it suffices. When it works.

No, because G isn't the data you're obtaining in your experiment.

I didn't say it was. I said it was the factor by which the data you're obtaining are massaged.

No, I don't think he does that, but he does observe the polls and note which have been successful in the past and weight those more heavily. He tries to "correct" for things he thinks are deficient in the polling executions (bias, sample, date, etc.) and weights the data he has accordingly. But that's subjective as well. It's subject to expertise -- Silver clearly isn't stupid and is actually very good at what he does. But, it's still subjectively altered data as an input. He's using stats on his opinions.

If the weighting is in response to prior election results, in what sense is it subjective? If I adjust the gravitational constant of my physical model in response to experiment results, is that subjective?

Are you unaware of error bars? Silver gave central guesses, but his model quite often encompassed both results inside the error margin. Again, his calls on individual states were his opinion on the model outcome, because the actual outcome of his statistical model on his choices of data had error bars that weren't as precise. You can choose to ignore that because his headline guess was often right (he's had a few bombs, as well), but the model you're claiming is the science is not at all as confident. This is available for looking at (one other thing I credit Silver for, he's discloses pretty well).

Are you saying that science is always confident and precise? I think you know better, but that really sounds like the argument you're making here.

Just like when a scientist opines on philosophy they're doing philosophy and not science, when a historian opines on current events, he's not doing history. I have a lot to say on music, but that doesn't make me a musician (no truer benefit to the world).

You're doing nothing here but contradicting me, so I don't know what else to do but contradict you right back. Current events are absolutely within the domain of the historian, because the historical processes that historians study are not confined to the past and do not suddenly cease to exist at the present. When a historian of, say, labor relations comments on a contemporary labor dispute, that's not like a chemist commenting on philosophy - it's like a chemist commenting on more chemistry.

 

[Maxperson]

I'm going to take a stab in the dark and guess that you don't read many horoscopes? The key element of horoscopes and prophecies, the trick that makes them work, is that they make vague predictions which can can be interpreted after the fact as conforming to whatever happened, and are thus difficult to falsify. Whatever else you may say about political prognosticators, they do not do that. We can say, definitively, what Nate Silver was right about and what he was wrong about. That clarity is in falsification conditions the sine quae non of science. So to compare a prediction which has such clarity with one which doesn't is deeply unfair and, when the whole point of this discussion is "what is science?", actively misleading.

I'm going to take a stab in the dark and guess that you only read online and/or newspaper horoscopes. A real horoscope done by a serious astrologer involves calculations that involve figuring out where the stars, planets, sun and moon were when you were born. My parents were hippies in San Francisco and my mother had a horoscope done when I was born. It was much more complex than the ones that you find online or in a paper.

 

[TheCosmicKid]

I'm going to take a stab in the dark and guess that you only read online and/or newspaper horoscopes. A real horoscope done by a serious astrologer involves calculations that involve figuring out where the stars, planets, sun and moon were when you were born. My parents were hippies in San Francisco and my mother had a horoscope done when I was born. It was much more complex than the ones that you find online or in a paper.

...

Read the paragraph you quoted again.