What would be some good metics to evaluate RPG rules/systems?

Guest 6801328 · Jun 11, 2019

When an interesting sounding RPG comes along, I struggle to get a sense of what the rules are like. Reading reviews I get a good sense for the genre and mood, but what I really want to know is where the rules are in terms of complexity and...well, I'm not really sure what the axes are. I think about the differences between (to grab a few random examples) systems like Dungeon World, various editions of D&D, MERP (!!!), Fantasy Flight Star Wars, and some others and although I have preferences I have trouble specifying how you would measure the differences.

What I'd LOVE is a 2D scatter plot, where games are placed (by consensus?) on two axes. I suspect one axis would be "complexity". What other axis would provide the most useful information? Maybe something about adherence to strict rules, versus narration/interpretation? Although I suspect that axis would correlate pretty strongly to general complexity.

Alternatively (or additionally) having two axes of "chargen complexity/options" and "gameplay complexity" might be interesting.

And then, in addition to numerical ratings, maybe there are some categorical tags that help explain what the rules/systems are like. But, again, I'm not sure what they are.

An analogy of what I'm looking for might be how we shop for cars (size, type, seating, engine, performance, etc.) or houses (price, zip code, style, # of rooms, etc.). Those axes don't give *all* information, but they help you narrow down your search.

What would the equivalent be for RPGs?

Thoughts?

Morrus · Jun 11, 2019

It's not like a car. It's much more subjective and much more of an art. You can directly and objectively measure a car's horsepower or weight, but you can't do that with an RPG.

I don't think anything can substitute for actually trying the thing. All the graphs in the world won't give you a sense of how you feel when you play the game.

lowkey13 · Jun 11, 2019

*Deleted by user*

hawkeyefan · Jun 11, 2019

The core mechanic may be a good indicator of how the game will play, but I don't think that's something that can be put on an axis in a graph. I agree with Morrus in that it's much more subjective than that.

But there are some things that do help understand how a game will play.....d20 based, d6 based, percentile dice....those give an idea of things. Then there are systems like Powered by the Apocalypse or FATE or Savage Worlds, where there are multiple games, but all share a good deal of mechanics or gameplay design.

But the kind of metrics you're looking for....not sure if there are any suitable ones.

ART! · Jun 11, 2019

boardgamegeek has a pretty good rating system for boardgames. Here's a random sample of their header for a board game:

Their sister site rpggeek does NOT have similar headers for RPGs, but does have the same kind of 1-10 user ratings.

If you do an advanced search for a board game, there are lots of mechanics and other ways to narrow things down. These are screencap samples of the three criteria/categories:

Screen Shot 2019-06-11 at 11.09.59 AM.png

Screen Shot 2019-06-11 at 11.10.12 AM.png

If you do an advanced search for an rpg, there's much less of that, and more about genre and setting. These are screencap samples of the three criteria/categories:

Screen Shot 2019-06-11 at 11.10.46 AM.png

Screen Shot 2019-06-11 at 11.11.02 AM.png

I don't have time to draw any conclusions from that.

uzirath · Jun 11, 2019

A major challenge for this endeavor is that one's actual play experience is not always tightly linked to the rules. This is not to say that system doesn't matter — it definitely has an impact — but RPGs are so complicated and have so many aesthetic dimensions that different groups run them entirely differently.

For example, I recall one of the great dungeon master's of my youth would ask everyone to show up with a description of their character on a single 3x5 index card. No stats, no equipment lists, just a bit of backstory fluff and a description of their basic talents. This was 1e AD&D. It sure didn't feel like a lot of other AD&D tables. Similarly, I've played D&D games in other editions that were all combat and others that were mostly role-playing with precious few die rolls in any given session. I've played at GURPS tables where every optional rule seemed to be in effect with scores of books available for consultation and where the tactical combat system was really its own hyper-complex battle simulation. And I've played at GURPS tables that were minimalist with simple wildcard skills, zero combat, and no books at all.

Again, this isn't to say that the rules systems themselves might not be measurable in some manner, and compared against each other, but I wonder how much utility this would have in terms of predicting the actual experience of play? A narrative review may have more utility because it won't be constrained by the elements that are being measured, but reviewers may need to do a better job at describing the crunch

ART! · Jun 11, 2019

You could have users rate a game overall on some scale, and/or rate things like layout, indexing, clarity, organization, etc.

You could list if there's randomization, and if so what is used for randomization (generic 52-card deck, proprietary cards, dice - of what kind and number, etc...). Then you could list general mechanics like "roll under", "roll over", "count successes", "degrees of success", "exploding dice", etc.

Umbran · Jun 11, 2019

Don't ask, "What would be a good metric?"

Ask instead: "What matters to you about a game?" Not just as a thought exercise - Make an actual list of things that matter to you.

We can try to separate what of those things are GM/group dependent, and what's rules-dependent, and what measures we could put that could be useful.

Metrics should be chosen to answer specific questions.

Beleriphon · Jun 11, 2019

Umbran said:
Metrics should be chosen to answer specific questions.

I think there are a few good metrics for rules that are universal. Organization, clarity of writing, really anything one would use to evaluate whether any reference book is good at doing what it is supposed to do.

Guest 6801328 · Jun 11, 2019

Morrus said:
It's not like a car. It's much more subjective and much more of an art. You can directly and objectively measure a car's horsepower or weight, but you can't do that with an RPG.

I probably shouldn't have used the word "metric" because it suggests something objective, whereas I'm imagining a value that's subjective, and/or maybe an aggregate/consensus value.

But I do think think there are things that can be evaluated this way, such as the examples I gave.

Or maybe it's sort of a correspondence analysis, where you would say the game in question has these features like games A, B, and C, and these other features like games D, and E, but this other feature like game F.

I don't think anything can substitute for actually trying the thing. All the graphs in the world won't give you a sense of how you feel when you play the game.

The same could be said for driving cars and visiting houses. And yet time is finite.