A discussion of metagame concepts in game design

[Shasarak]

It's a method of data analysis, yes, and, as such, can be used to do science. It is not science. Doing stats does not make what you're doing science.

You wouldn't actually have to use stats for that experiment, either. Find a reasonable similar cohort of subjects and have a control group, then compare results.

Comparing results is exactly what stats is doing.

Medicine is another place where bad use of statistics is rampant.

If you think that it is just bad statistics in medicine to worry about then you have another think coming.

Still it is a process, even the best (worst?) efforts of the tobacco industry could not hide the problems with smoking for example.

 

[TheCosmicKid]

And to go back to the tangent that spawned this tangent: I don't think that the default gameworlds of fantasy RPGs assume anything about what the results of such careful measurements would be. They don't need to, as it's the nature of most fantasy RPGing that common sense tropes - "unsupported objects fall", "winter is cold", etc - are sufficient to permit players to sensibly declare actions for their PCs, and GMs to adjudicate the results of those action declarations.

Attempts to apply realistic physics to falling damage rules are particularly notorious.

 

[TheCosmicKid]

Actually, I'd be quite happy with the complete demise of the p-value. There are really only a handful of good cases for frequentist stats (RF being one), but you could do with Bayesian techniques (harder, but less misleading).

But, as [MENTION=92239]Kobold Boots[/MENTION] put it, you're drifting away from the mainstream there.

Science is the scientific method.

That's tautological. What's the scientific method? Why doesn't observation-based statistical prediction qualify?

 

[Maxperson]

What's the scientific method? Why doesn't observation-based statistical prediction qualify?

"Steps of the Scientific Method
Make an Observation
Scientists are naturally curious about the world. While many people may pass by a curious phenomenon without sparing much thought for it, a scientific mind will take note of it as something worth further thought and investigation.

Form a Question
After making an interesting observation, a scientific mind itches to find out more about it. This is in fact a natural phenomenon. If you have ever wondered why or how something occurs, you have been listening to the scientist in you. In the scientific method, a question converts general wonder and interest to a channelled line of thinking and inquiry.

Form a Hypothesis
A hypothesis is an informed guess as to the possible answer of the question. The hypothesis may be formed as soon as the question is posed, or it may require a great deal of background research and inquiry. The purpose of the hypothesis is not to arrive at the perfect answer to the question but to provide a direction to further scientific investigation.

Conduct an Experiment
Once a hypothesis has been formed, it must be tested. This is done by conducting a carefully designed and controlled experiment. The experiment is one of the most important steps in the scientific method, as it is used to prove a hypothesis right or wrong, and to formulate scientific theories. In order to be accepted as scientific proof for a theory, an experiment must meet certain conditions – it must be controlled, i.e. it must test a single variable by keeping all other variables under control. The experiment must also be reproducible so that it can be tested for errors.

Analyse the Data and Draw a Conclusion
As the experiment is conducted, it is important to note down the results. In any experiment, it is necessary to conduct several trials to ensure that the results are constant. The experimenter then analyses all the data and uses it to draw a conclusion regarding the strength of the hypothesis. If the data proves the hypothesis correct, the original question is answered. On the other hand, if the data disproves the hypothesis, the scientific inquiry continues by doing research to form a new hypothesis and then conducting an experiment to test it. This process goes on until a hypothesis can be proven correct by a scientific experiment."

Because predictions don't follow the scientific method. There is no carefully designed and controlled experiment, and that's one of the most important steps in the process. It even says so!!

 

[TheCosmicKid]

Because predictions don't follow the scientific method. There is no carefully designed and controlled experiment, and that's one of the most important steps in the process. It even says so!!

That's the grade-school presentation of the scientific method. In real sciences of many varieties, "experiments" have to take the form of making further careful observations of the world and assessing whether they conform to a prediction made by the hypothesis. For example, controlled and repeated experiments are not usually possible in astronomy: if we want to test, say, the prediction made by general relativity about gravitational lensing, we have to wait until a solar eclipse and then measure the apparent positions of the stars around it. Make no mistake: these "natural experiments" are not as good as controlled experiments in a laboratory. There is greater risk of the result being a consequence of an uncontrolled variable, and as far as repeatability goes we're at nature's mercy. We have to be more careful, and cannot be as certain. But that does not mean that no science can come of it. It seems absurd to declare that the Eddington-Dyson observation of the 1919 eclipse, one of the most dramatic and famous experimental confirmations of a hypothesis in the history of science, was "not scientific".

 

[pemerton]

That's the grade-school presentation of the scientific method.

Thanks you!

As well as the examples you point to (astronomy) we could point to aspects of biology (Darwin didn't do experiments - he made very careful observations and conjectured the best explanation for them given the constraints he took to be applicable) or demography/public health (no one goes about spreading diseases or polluting water supplies to try and determine the effect on life expectancy, birth rates, and other aspects of population health), etc.

Beginning with the caveat that while I have taught and also supervised in philosophy of science it's not my main gig, I would start any account of the scientific method with the centrality of careful observation and measurement. Experiments are a device for facilitating this, but not the only way.

In the fields that are my main gig - law, political and social philosophy, philosophy of language - one of the most common causes of bad work is (in my view) a lack of careful observation - which mostly manifests itself in treating common sense about one's own social situation or one's own linguistic usage as if were a more-or-less necessary feature of such things. (Even with careful observation - eg attention, when doing social theory, to the widest possible range of human social formations - these disciplines still will not count as scientific for the most part, as they typically don't involve measurement.)

 

[Sepulchrave II]

[MENTION=16814]Ovinomancer[/MENTION], re: reification. I have pondered your posts, and the point is well made. I had never before encountered the term in the context in which you used it. As it happens to be one of my favorite words, I felt it necessary to do some digging. I am no statistician, but I don’t need to be; I fully understand the fallacy of treating a model of a thing as the thing itself. But this line is more interesting:

Statistical modelling isn't science, though, it's horoscopes with math.

Because 400 years ago, astronomy was that literally. Like alchemy, it needed to shake off the woo-woo, and enter respectability.

My sense is that with economics, sociology, psychology, we are observing phases in nascent sciences; the disciplines are still struggling to cohere. Psychology is hardening – largely because of neuroscience – but it still has a long way to go. The traditional physical sciences are more established, and have had longer to iron out their wrinkles – perhaps that is why the soft edges of the soft sciences are frustrating to those who are grounded in the hard.

I would submit that history is one of the youngest sciences; historians seem singularly resistant to any kind of systematic inquiry and display an almost paranoid avoidance of mathematics. Bayesian reasoning is beginning to appear at the fringes; it is not being well-received – because history professors are ignorant and lazy. My evidence is purely anecdotal.

And what I think matters not one whit. If a discipline really is a science, it will self–correct and improve, and demonstrate its value – such is the virtue of science; if not, it goes the way of homeopathy, confined to a few fringe cranks who seek ever-deeper meanings in its arcane formulations.

 

[VisanidethDM]

I raised this issue on several occasions with Mike Mearls, but I have to think he didn't understand me. He'd give an answer that sounded like he cared about the concern but then we got the game as written. So I think maybe he thought I was asking about X when in fact it was about Y. Otherwise I hope he would have not been so encouraging.

I think your problem is that you need to look for really, really simple games that limit player choices as much as they can. Everytime a player gets to choose a different type of action you basically create a potential disconnect between player and character because the character may not be as informed as the player on the context of that action.
What you want is a system where you describe your action, roll a dice, and you get to describe what happened based on the idea that the character is hoping for the best result possible every time and how close he gets is based on how well he rolls.

What about Pf2e? Do you think it will do this particular issue better? I've been following your posts about that game but not sure I can be sure. Realize too that I only need a workable subset of a system. If I had to ban a few classes I could easily live with that. Fighter, Wizard, Rogue, and Cleric are core and I wouldn't want to ban those.

If Action Surge (the idea that going overboard with effort will mean you won't be able to do it again, think of "I'm doing 5 more reps on this exercise but this means I won't be able to do another series") is already triggering your metagame threshold I think PF2 isn't going to be a game for you.

 

[VisanidethDM]

In an earlier thread, I got one of the developers to confirm that every hit on an attack roll corresponds definitionally to some sort of physical impact, so Hit Points should be less abstract in PF2 than they are in D&D 4E or 5E.

I know that it's tangential to the topic at hand, but it gives some indication as to their stance on overtly gamist mechanics.

A character being able to survive 10-15 direct hits a day and becoming capable of withstanding inordinate amounts of punishment as he becomes more competent IS the overtly gamist mechanics.

 

[Maxperson]

That's the grade-school presentation of the scientific method. In real sciences of many varieties, "experiments" have to take the form of making further careful observations of the world and assessing whether they conform to a prediction made by the hypothesis.

All hypothesis is prediction in the scientific method. You are conflating prediction of something specific that you are trying to prove via controlled experimentation and analysis, and prediction of something specific and involves no controlled experiment at all(an election).

For example, controlled and repeated experiments are not usually possible in astronomy: if we want to test, say, the prediction made by general relativity about gravitational lensing, we have to wait until a solar eclipse and then measure the apparent positions of the stars around it. Make no mistake: these "natural experiments" are not as good as controlled experiments in a laboratory. There is greater risk of the result being a consequence of an uncontrolled variable, and as far as repeatability goes we're at nature's mercy.

This is a controlled experiment. The control isn't as great as the lab, but it's there. We can know with incredible accuracy when solar eclipses will happen. I watched one earlier this year or maybe it was last year. I can't remember. I was able to go outside exactly when it happened because of repeatability of experiments in the past. Yes, we're at nature's mercy on when we can repeat the astronomical experiment, but repeatable it is.

We have to be more careful, and cannot be as certain. But that does not mean that no science can come of it. It seems absurd to declare that the Eddington-Dyson observation of the 1919 eclipse, one of the most dramatic and famous experimental confirmations of a hypothesis in the history of science, was "not scientific".

Whereas you cannot repeat an election. Even going to a second term, so much has changed that Trump's second attempt at election cannot possibly be viewed as a repeated experiment. There is no science involved in these sorts of predictions. You might as well get another Paul the Octopus to do it.