What would be some good metics to evaluate RPG rules/systems?

Some commenters have raised analogies to summarized information about cars and houses. People generally buy one car at a time, out of many options, and one house at a time, out of several or many options. So is this proposed system also meant to inform purchase choices?

A system for comparing TRPGs might be useful for informing choices about registering for convention games. I might prefer a game along these lines: "roll 1d6 if you're in unfavorable situations, 2d6 for average tasks, 3d6 if you're well-trained, well-equipped, or in favorable circumstances." I might dislike games along these lines: "roll 1d100 for accuracy, applying modifiers for characteristics, skill ranks, equipment category, and other factors; then roll 3d6 for power, also with modifiers; compare your Accuracy outcomes to the target or task's Elusive rating (also rolled on d100) and your Power outcome to the target or task's Resistance rating (a fixed number from 0 to 25). If I can sign up for either the 10AM Saturday game of Hackmaster, or the 10AM Saturday game of Doctor Who, then I might enjoy knowing which game has more moving parts in its action resolution system.

Yes, metrics should have actual numbers. Sometimes, failing hard numbers, rankings are useful. I could not tell you the numerical Scoville ratings of the hot, medium, mild and supernova salsas at my local tacqueria; but I can accurately rank them from most to least spicy. (That ranking isn't the alphabetical order I used, when listing the categories, just now.)

Metics, on another hand, were foreigners living in an ancient Greek city who had some of the privileges of citizenship.

Elfcrusher said:

When an interesting sounding RPG comes along, I struggle to get a sense of what the rules are like. Reading reviews I get a good sense for the genre and mood, but what I really want to know is where the rules are in terms of complexity and...well, I'm not really sure what the axes are. I think about the differences between (to grab a few random examples) systems like Dungeon World, various editions of D&D, MERP (!!!), Fantasy Flight Star Wars, and some others and although I have preferences I have trouble specifying how you would measure the differences.

What I'd LOVE is a 2D scatter plot, where games are placed (by consensus?) on two axes. I suspect one axis would be "complexity". What other axis would provide the most useful information? Maybe something about adherence to strict rules, versus narration/interpretation? Although I suspect that axis would correlate pretty strongly to general complexity.

Alternatively (or additionally) having two axes of "chargen complexity/options" and "gameplay complexity" might be interesting.

And then, in addition to numerical ratings, maybe there are some categorical tags that help explain what the rules/systems are like. But, again, I'm not sure what they are.

An analogy of what I'm looking for might be how we shop for cars (size, type, seating, engine, performance, etc.) or houses (price, zip code, style, # of rooms, etc.). Those axes don't give *all* information, but they help you narrow down your search.

What would the equivalent be for RPGs?

Thoughts?

Click to expand...

If you are looking at cars as an analogy, I think basic information on a table so you can compare systems and settings easy (System, Cost, Complexity, Style//Genre, Rating/Ranking (if applicable), etc).

Elfcrusher said:

See, I actually think that "rulings" are...or can be...a good thing, and it's one of the things I'd like to know about a game. To what extent do the rules attempt to specify outcomes for every scenario, versus how much is left to interpretation?

Click to expand...

That's not a matter of clarity, but of scope or completeness, and not an interpretation or a ruling, but simply adding - formally or informally - to the game.

For instance, if a game includes no prices, stats, or rules for weapons, then armed combat might be outside its scope - maybe it's all about boxing, IDK. The rules it does present might be clear enough that no rulings or interpretations are called for, everyone who reads the rules can play the game without confusion or argument over what the rules mean. But, when a player decides his character will hit someone with a folding chair, or try to buy a gun, the GM will either deny him, or add to the game.

Now, if that game was actually supposed to be a film noir detective RPG, the lack of any rules covering firearms would be more a matter of the rules being incomplete, than of them being unclear.

Tony Vargas said:

For instance, if a game includes no prices, stats, or rules for weapons, then armed combat might be outside its scope -

Click to expand...

I know you said "might", but...Dungeon World?

Sure, some weapons have stats, but there's no list. And definitely no prices. You do damage based on your class, regardless of how you decide to describe your weapon. Dual-wielding rolling pins? Same damage.

That's the kind of thing I'd like to know about in evaluating a game.

Again, maybe the solution is not to quantify things, but to rate games on a bunch of things in terms of other games they are like?

Elfcrusher said:

I know you said "might", but...Dungeon World?

Sure, some weapons have stats, but there's no list. And definitely no prices. You do damage based on your class, regardless of how you decide to describe your weapon. Dual-wielding rolling pins? Same damage.

Click to expand...

That's still rules for using weapons, just less detailed ones.

And it's hard to say a game is X level of detailed. For instance, 1e AD&D had a more detailed list of weapons, with more detailed stats than 3e, but 3e had more detailed rules for using them.

How do you rate those differences?

Again, maybe the solution is not to quantify things, but to rate games on a bunch of things in terms of other games they are like?

Click to expand...

What if a game were to (gasp) do something differently than other games?

In the 80s, 'Core Systems' seemed to me a big part of the industry paradigm. The idea being, if you liked, say, CoC, you'd easily be able to play other BRP based games. The culmination was probably GURPS.

Point being, that's kinda what you're looking for: new games that are enough like existing games you're already familiar with and like.

I work in software development.

There is a general sense in the industry, that for two given methods for solving a problem, the one that is lower complexity tends to be the better solution. We are certainly encouraged to right rules that reduce complexity according to very similar measures as some have proposed regarding RPG rules - length of the rules, readability, number of branching paths, number of subsystems, depth of hierarchies, number of cross references, number of meta-rules, etc. The trouble is that it turns out that measuring any of those things in a meaningful manner is really hard, and often two metrics are in tension with each other - for example, you can make the rules more compact, but only at the cost of reducing readability, increasing cross references, and/or increasing the number of meta-rules. So then there is the problem of how you would weight the tradeoff. And then it turns out that things like space, complexity, and processing time are also trade offs, so that you can increase "speed of play" but only at the cost of longer and more complex rules, such as increasing the number of tables.

So in short, if you could solve this problem in an academic sphere with relatively little economic impact, then you could solve this problem in an academic sphere with enormous economic impact, and probably set yourself up for a very prestigious prize in mathematics or economics. It would be certainly the sort of thing that could get you nominated for a Nobel prize or the like.

Tony Vargas said:

Point being, that's kinda what you're looking for: new games that are enough like existing games you're already familiar with and like.

Click to expand...

That's quite a leap, and not even a remotely accurate conclusion. I asked the question whether rating games on their similarity (on various dimensions) to existing games would be a useful thing. I didn't say that I'm looking for new games that are like existing ones.

In fact, in some cases knowing that new game X is really far away from existing game Y, on nearly all dimensions, would be a positive sign for me.

That said, you raise a good point: if a game is totally unlike existing games, how do you evaluate it using that method.

Celebrim said:

I work in software development.

There is a general sense in the industry, that for two given methods for solving a problem, the one that is lower complexity tends to be the better solution. We are certainly encouraged to right rules that reduce complexity according to very similar measures as some have proposed regarding RPG rules - length of the rules, readability, number of branching paths, number of subsystems, depth of hierarchies, number of cross references, number of meta-rules, etc. The trouble is that it turns out that measuring any of those things in a meaningful manner is really hard, and often two metrics are in tension with each other - for example, you can make the rules more compact, but only at the cost of reducing readability, increasing cross references, and/or increasing the number of meta-rules. So then there is the problem of how you would weight the tradeoff. And then it turns out that things like space, complexity, and processing time are also trade offs, so that you can increase "speed of play" but only at the cost of longer and more complex rules, such as increasing the number of tables.

So in short, if you could solve this problem in an academic sphere with relatively little economic impact, then you could solve this problem in an academic sphere with enormous economic impact, and probably set yourself up for a very prestigious prize in mathematics or economics. It would be certainly the sort of thing that could get you nominated for a Nobel prize or the like.

Click to expand...

Again, though, I'm not looking for an objective metric. 10 software engineers with sufficient experience and knowledge would be likely to give a given piece of source code a somewhat similar rating for compactness and elegance. Exactly the same value? No. But in the same ballpark.

Likewise, if 15 reviewers give Chargen Complexity an average of 1.7 out of 5, that's good enough for me.

Anyway, I'm not sure the idea I'm proposing has any merit, but it's something I've wondered about.

lowkey13 said:

10. And .... number of words not found in the OED per page.

Click to expand...

Objection!

This metric would wildly skew the position of my fantasy heartbreaker, inspired by nostalgia for my time gaming in Montreal with a Quebecois group even though the only French I knew was what I learned in school, not really paying attention in French class.

Les Dungeonnes et Les Dragóns, written in what I lovingly dub Satyrninian Canadian is full of words found in neither the OED, nor whatever dictionary the French use.

Elfcrusher said:

I didn't say that I'm looking for new games that are like existing ones.

Click to expand...

You didn't say it, may not believe it, but the rating systems you're thinking about - and your reactions to what's been proposed as possible ratings - make me think that's what you're looking for...
... or, at least, what you'll end up finding, by looking in the manner that you seem to be leaning towards.

That said, you raise a good point: if a game is totally unlike existing games, how do you evaluate it using that method.

Click to expand...

You'd end up evaluating it negatively (in the sense of judging it to be bad, or in the sense of only being able to define what it's not), or even as not being a game at all.