Rules Heavy v. Rules Light experiment - is it feasible?

[nothing to see here]

So, if this were a competition, we'd have problems. So, we'd have to do a double-blind study instead. So, to do this more accurately, we'd need newbies who didn't understand gaming at all (or just the basic concepts of it), and then sit them in a room with some pre-generated characters and go through the dungeon.

Without the DM/CK attempting to "beat" anyone else, it would be a more valid test. And yes, statistically, we'd need to do this with like 40 groups each to be scientifically accurate, and even then there'd be a number of other factors that could skew the results.

I guess this all boils down to one question - what did Dancey hope to achieve with his test? We've already ascertained within about 20 posts on here that such a test would be faulty on many levels.

This system would work. And, my guess is, this is the kind of study Dancey was referring to.

Of course if you're looking to sample from 'newbies who don't understand gaming at all' there are probably better places to look than GenCon.

 

[nothing to see here]

Which makes this (and other) discussions more or less a pointless endeavor.

It's not completely hopeless. Just because something cannot be defined conclusively does not mean it cannot be defined at all.

All somebody needs to do is come up with a working definition of "Rules Lite", develop a set of critiera and a matrix by which different systems can be found to be more or less "rules lite"...and use that matrix for the purposes of the study. Such a study would produce some sort of results.

Now, of course, it is anybody's and everybody's right to dispute the validity of the original definition of "rules lite", and substitute one of their own. The same goes for the methodology by which you break down different systems. Such debates are the meat and drink that keep many an academic journal alive.

But for people who agree with the stated definition of "Rules Lite" the study would still have merit.

What we need is somebody will the (insert juvenile reference to male genitalia here) to step out and post A) a definition of "rules lite" and B) a set of criteria by which to compare systems to this definition...understanding fully that not everyone will agree with his/her choices.

 

[jmucchiello]

I still say pages of "rules" is the best test for light/heavy. If something comes up and you ahve to look it up, how much space do you have search through to find the answer. C&C cannot be rules lite because it has hundreds of spells. Each spell is its own rule. Each monster has its own rules (unlike 3e where common rules are separated out). Each magic item is its own rule. Note that for some games there lot of pages but many of those pages are setting so the book page count is not at issue, but the rules page count.

Risus is 6 pages. It is definitely rules-lite. After that, it's hard to say.

 

[Banesfinger]

Another thing to consider (lite/heavy) that this test would NOT cover (since it is only a dungeon crawl) character updates.

How much effort is involved in updating (i.e., character makes a level) a PC?

For example, do you have to spend 1 hour to update?

Does an increase in a primary stat cause a ripple effect in other areas (e.g., skills). Using this last example, many of you will say "that's no problem, my XL spreadsheet takes care of all that". However, try doing it by hand (manually), because computer programs, and calculators are definately a sign of rules "HEAVY".

 

[diaglo]

It's ironic that their doing an actual study is poo-pooed. Apparently anecdotes and preconceived perceptions are more than enough evidence for folks around here. "We don't need no steenkin' facts."

a bad study can do more harm than good

 

[diaglo]

You'd also need a volunteer sample who would be oblivious to what you were testing. Mere knowledge (or even pre-determined suspicion) of the variables to be measured would make the test a total write-off.

It's a common error in focus groups. People can know they're being observed and recorded. They can know the issues they will be expected to discuss. They can even come to suspect the parameters of the study IF this knowledged is gleaned totally within the confines of the test itself (in fact it's encouraged). However should any participant steps into the test knowing or suspecting what the people behind the glass are looking for, the focus group becomes useless...

...it's why people who work in public relations, marketing, or other fields prone to using social research are screened out of focus groups (which is a shame since, by my count, I could have made close to a thousand dollars in these studies already)...we spend our time not thinking about what the moderator is saying...and instead try to figure out what they're saying behind the glass.

true.

but you also have to figure out a measurement. and a grading system. and a whole bunch more.

saying "lets do a study" or even "i did a study" doesn't mean it is accurate nor true. nor reproduceable nor sound. nor a whole bunch of other things.

 

[Ourph]

Comparing the amount of time required to achieve certain in-game goals is not a valid way to measure rules-lite vs. rules-heavy. The time will be much more heavily affected by play-style than by rules complexity. If the C&C group sits around discussing what the bad smell coming from the left-hand corridor means for 30 minutes; or if the D&D group simply runs through the challenges, taking whatever AoO's come their way and hoping to soak the damage and reach the end of the dungeon just to finish first; the amount of time taken to "finish" will have absolutely nothing to do with the complexity of the rules for either game.

I think the better measure is what the Forge calls "handling time", i.e. - when a situation comes up in game for which the group doesn't know the rule/outcome, how long does it take to get an answer.

 

[Psion]

I think the better measure is what the Forge calls "handling time", i.e. - when a situation comes up in game for which the group doesn't know the rule/outcome, how long does it take to get an answer.

I don't even think that's a valid measure. Handling time is pointless if you acheive a result that leaves me dubious or wondering if the GM is playing favorites or feeling like I am being railroaded or shatters my SOD in half the time.

Of course I receognize that my sensitivity to these issues differs from others. But that's why you can't really make a case about "betterness" that holds any validity outside a group of players with similar tastes and values.

 

[Ourph]

I don't even think that's a valid measure. Handling time is pointless if you acheive a result that leaves me dubious or wondering if the GM is playing favorites or feeling like I am being railroaded or shatters my SOD in half the time.

Of course I receognize that my sensitivity to these issues differs from others. But that's why you can't really make a case about "betterness" that holds any validity outside a group of players with similar tastes and values.

When I say "get an answer" I mean, "get an official answer for a question that has a definitive rule". Situations that are handled by DM fiat, or situations where the group simply goes with what "feels right" would necessarily be excluded from the evaluation.

In order to do an accurate and statistically valid study of "handling time" you would really have to establish some very strict definition of what situations would be included and then do extensive monitoring of numerous groups to get a reasonably large data set. Personally, I can't imagine why anyone would bother with such an experiment (or who the heck you'd get to fund it ). Any conclusion about handling time is only useful for comparing handling time. Any attempt to use it to define "rules-lite" or "rules-heavy" is still going to come down to someone's arbitrary decision that an average handling time of X is the dividing line between the two. The measurement might be more exact, but the distinction is still going to be based on opinion, not fact.

 

[gizmo33]

"Handling time" doesn't take into account that a rules-light system doesn't even give answers for the same questions as a rules-heavy system, and I think this is a fundemental difference between the two.

I can create a rules-light system that says "flip a coin, and if you get heads, you solve the dungeon, tails, you die. The DM can add flavor text as appropriate." Clearly this system beats any RPG on the market (I hope) in terms of speed.

It's not clear to me what hypothesis is really being tested by the test being proposed.