D&D 5E What is Quality?

Ovinomancer

No flips for you!
You are effectively asserting that the only qualities that exist are intrinsic qualities. This is an inaccurate picture of quality, overall - extrinsic qualities still matter in the real world.
You've done a semantic goal post shift on quality, here. You've moved from a term describing a measure of excellence to a term describing a feature or characteristic. Please don't. Adding this kind of motte and bailey setup isn't improving the discourse.
 

log in or register to remove this ad

Morrus

Well, that was fun
Staff member
While I agree (but chose to ignore it), it can be pointed out that the image shows a plane that took all those hits and still flew. Making it a high quality plane for it's time and purpose.
No, a plane didn't take all those hits. Those are the hits received by hundreds of planes plotted onto one diagram. If a single plane took all those hits, there'd be nothing left of it.
 

FrogReaver

As long as i get to be the frog
@Maxperson got to the point I was skirting around earlier but that I didn't express very well. When we assess the overall quality of a thing, we are looking at an array of particular aspects in combination. Some of those aspects can be objectively measured -- reliability or fuel efficiency of a vehicle, or the nutritional value of a hamburger, or its cost. Some of those aspects are subjective -- aesthetics of a car's design, comfort of the seats, or the appeal of the smell of the burger.

What is also subjective is which array of aspects we value and weigh more highly than others.

Expert opinions from trained and informed sources are invaluable to gauge particular aspects of a thing, and those opinions might even convince us to value more highly aspects of a thing that we hadn't considered important before, but we can say for ourselves -- and not for anyone else -- if reliability and safety are better indicators of quality than performance and speed.

In the case of an RPG, there are I think few aspects that are objectively measurable. We can, like many have done in this thread, talk about what aspects of a game we value that informs our assessment of the quality of the game, but we're unlikely to come to consensus. We can suggest that high popularity is an objective measure that indicates that the game has many aspects that meet the criteria of a lot of people, but that won't convince everyone that it's a quality game.
I’d be more willing to entertain the idea that popularity implies some minimal threshold of quality.
 

Oofta

Legend
No, a plane didn't take all those hits. Those are the hits received by hundreds of planes plotted onto one diagram. If a single plane took all those hits, there'd be nothing left of it.
I may not have said that correctly, but it can still an indication of quality IMHO.
 


Bah, scrolling back for information? Sounds like a job for the objectivists. It’s so much simpler over here with the subjectivists, where all I have to do is feel!
For bizarre historical reasons (mostly a group taking on the name as though naming themselves that actually adds credence to their position), that specific term doesn't tend to mean 'person actually being objective/preferring objectivity.'
I'm not sure what you base this last point on. 4e had a loud group of detractors (including Mearls) but you have no evidence at all about what the majority experience of it was. In fact, Wizards at the time said it was the best-selling edition ever, making it the second most popular RPG of all time. By the arguments posed in parts of this thread that makes it the second best RPG of all time.
This gets thrown around so frequently, yet no one ever can bring up the reference so we can see what was actually said, and thus what constraint there were on the claim. Was it the best selling edition at 6 months out? What specific aspect were they measuring (units vs. dollars, gross vs. net)? Did they mean the gamebooks (or even just the core gamebooks, which would be notable since a lot more of 4e counted as core than other editions) or total D&D product at the time? Without that context, the statement is of extremely limited use.
And, yes, it is absolutely useful to suppose that some judgements are superior. Because if we don't, then we accept bad faith arguments as valid as well as simply uniformed judgements as equally valid. The notion that all judgements are equal gets right back to the whole anti-intellectual thread that pollutes any conversation like this.

Engaging in thought exercises over the subjective nature of perception doesn’t necessitate the acceptance of all views due to that nature. Judgements can be born of ignorance or inexperience or, hell, even dishonesty. They can be measurably harmful and deserve their disavowal by the masses.
And I certainly understand the view that engaging those exercises is unproductive or frustrating or abstracted to the point of uselessness in ordinary discourse. But anti-intellectual against the alternative of accepting agreement in lieu of true demonstrability? You’d have to take the fight back to Descartes to prevent that from popping up. And this:
Only works for as long as you want to ostracize the very inevitable contention.
I find individual on nerd-centric online forum framing their* position as in defense against anti-intellectualism right up there with naming one's group 'the objectivists.' It is wrapping ones own position up in a term we all gravitate towards.
*Yes, I know it was actually Morrus who inserted the term.

There are objective truths (such as nutrition for a burger, maybe quality of paper and binding for a TTRPG rulebook) and also informed and uniformed judgements (someone can be genuinely wrong about what a game ruleset includes). Those are mostly beside the point to the discussion. The reason objective vs. subjective is used here so much is because many-to-most of the things being used to judge TTRPGs aren't going to have objectively better outputs unambiguously able to be placed on an ordinal scale. If that makes the threshold for objectivity unreachably high, so be it (and thus maybe the term best not be used in the situation). Plenty of other things can't 100% be objectively stated to be higher quality, but there can be a shared general consensus that something is important to a product, and there be relative universality in agreement (especially if something is egregiously bad at something, like a book with unreadable font or genuinely confusing page layout). Much of the rest of things may have objectively measurable qualities, but even then there is a subjective argument as to whether that quality is an unequivocable aspect of product quality.
Is a Ferrari or a Kia a more quality automobile? I see a lot more Kia’s around than Ferrari’s. Does that mean Kia is better quality than Ferrari?
I have ridden in a Ferrari*. It was loud, uncomfortable, and hot (AC could not keep up), and the headroom left much to be desired. Depending on how you measure 'takes you from point A to point B' (do you include measures of reliability and repeatability? etc.), it may not even be good at that primary function. That said, you are not wrong that Kia's popularity is not a measure of quality, any more than a Ferrari's desirability.
*Friend rented one for another friend's bachelor party
Yes. My parents had an old grandfather clock in the house that didn't work either. They kept it, though, because it served a purpose other than telling the correct time twice per day.

450px-Survivorship-bias.svg.png
Going by the implied argument of the last example, I don't think either of these are really that apt of comparisons to the topic of RPGs. Leaving aside the watch/clock-as-jewelry/art argument, the primary function of a watch is very obviously to tell time (and something like losing minutes a close-to-inarguable deficit in that regard). The bomber has a few competing functions (flying in general, carrying bombs, accurately dropping bombs, continuing to fly while being shot-up), but in general the quality is again fairly hard to argue against (even if you can argue about relative value of the above qualities, or even potentially how to measure). For TTRPGs, the comparable measure is (IMO) -- 'can the ruleset be used by someone to play a roleplaying session?' With few exceptions -- Hybrid being maybe an RPG at all but maybe just someone's word salad that includes some RPG framing, and FASA's 1980s Master of the Universe rpg (where there are rules referenced which never actually show up in the rulebooks) -- all TTRPGs meet that standard and most of the qualities used in arguments about which ones meet the standard better than others not being objective or even semi-universally agreed-upon.

All this leads me back to my overall conclusion that RPGs are closer to movies or music than burgers or clocks -- they all do the basic necessary requirements to qualify as the thing, some but not all of the components of their quality can be measured unambiguously, and there is no consensus on comparing product A with strengths in X and Y to product B with strengths in Z and W.

Getting back to the primary thread topic, no, popularity isn't in and of itself a measure of quality. There may be a correlation, and when something is really popular it is useful to analyze it to see what quality-related qualities it might have (and it's always useful to remember that what you assumed would be the measures of quality might not be right, and actually this popular thing happens to have something going for it of which you hadn't thought). It's at best smoke, which implies fire but doesn't guarantee it.
 


Oofta

Legend
I probably should have phrased this whole thread differently. In retrospect it should have been "Can you determine quality of a TTRPG?"

I don't think there is a great way. For physical products, there are different measurements such as reliability for vehicles. Popularity alone doesn't do it, especially when you're comparing apples (Rolex watches) to oranges (Casio watches). As another angle on this take a look at wine. For the vast majority of people they just want a slightly alcoholic beverage that tastes good and they cannot tell the difference between a $20 bottle and a $200 bottle. The latter bottle is only higher quality if accept criteria that has largely been developed and can only be detected by people that want to justify high priced wine.

My premise is that while D&D has always been big in the TTRPG marketplace, since 5E was released a decade ago it's seen double digit growth is an indication of a quality product. It's the best measurement we have. As I said in my original post "From a business perspective minimal investment + continued popularity + year after year double digit growth = quality."

That doesn't mean it's perfect. Nothing is.
 

Maxperson

Morkus from Orkus
I’d be more willing to entertain the idea that popularity implies some minimal threshold of quality.
Popularity alone is often based on the intrinsic qualities. One of the things that does contribute to popularity, though, is brand name recognition/loyalty, which is something that D&D has in spades. Of course, that has nothing to do with the actual quality of the game. It applied to 2e, 3e, 4e and 5e.
 

LadyElect

Explorer
For bizarre historical reasons (mostly a group taking on the name as though naming themselves that actually adds credence to their position), that specific term doesn't tend to mean 'person actually being objective/preferring objectivity.'
I was aware of this, but I indulge the re-reappropriation anyway in part as an attempt at Rand erasure.

especially if something is egregiously bad at something, like a book with unreadable font or genuinely confusing page layout
Ah, a fellow Danielewski disliker?
 

FrogReaver

As long as i get to be the frog
Popularity alone is often based on the intrinsic qualities. One of the things that does contribute to popularity, though, is brand name recognition/loyalty, which is something that D&D has in spades. Of course, that has nothing to do with the actual quality of the game. It applied to 2e, 3e, 4e and 5e.
Fully agree, but I do think the basic product needs to meet some minimum quality threshold or external factors like brand name recognition aren’t enough to maintain popularity.

Not saying that minimal level would be considered ‘good quality’. Maybe it’s ‘not terrible quality’.
 

Aldarc

Legend
Going by the implied argument of the last example, I don't think either of these are really that apt of comparisons to the topic of RPGs. Leaving aside the watch/clock-as-jewelry/art argument, the primary function of a watch is very obviously to tell time (and something like losing minutes a close-to-inarguable deficit in that regard). The bomber has a few competing functions (flying in general, carrying bombs, accurately dropping bombs, continuing to fly while being shot-up), but in general the quality is again fairly hard to argue against (even if you can argue about relative value of the above qualities, or even potentially how to measure). For TTRPGs, the comparable measure is (IMO) -- 'can the ruleset be used by someone to play a roleplaying session?' With few exceptions -- Hybrid being maybe an RPG at all but maybe just someone's word salad that includes some RPG framing, and FASA's 1980s Master of the Universe rpg (where there are rules referenced which never actually show up in the rulebooks) -- all TTRPGs meet that standard and most of the qualities used in arguments about which ones meet the standard better than others not being objective or even semi-universally agreed-upon.

All this leads me back to my overall conclusion that RPGs are closer to movies or music than burgers or clocks -- they all do the basic necessary requirements to qualify as the thing, some but not all of the components of their quality can be measured unambiguously, and there is no consensus on comparing product A with strengths in X and Y to product B with strengths in Z and W.
The point of the picture was not about the quality of bombers, but, rather, as a fairly well-recognized illustration of survivorship bias. The argumentation that "if the majority of people using a product works for them then it is a quality product" is a form of survivorship bias, as the argument ignores those who have stopped using the product because it doesn't work for them. The argument centers around a form of selection bias. It only selects for the majority of those whom the product works for. To be clear, I don't think that a TTRPG needs to work for everyone, but I think we should also identify survivorship bias when we see it. A better set of questions may involve asking who D&D 5e was designed for and who are the individuals and groups who continue using it and why?

I probably should have phrased this whole thread differently. In retrospect it should have been "Can you determine quality of a TTRPG?"
5e works for you. Why does it work for you whereas other games may not? What design qualities does 5e have that make it work for you or that you like? What does it empower you to do as a player or a game master? Does it deliver the experiences that it is designed for and how? How is the writing, production quality, and overall value of the product? I think that there are plenty of metrics you can talk about with 5e as a quality product without ever once needing to appeal to its popularity or growth.
 
Last edited:

Umbran

Mod Squad
Staff member
This is a graphic created showing the distribution of battle damage to the B-25s that returned to base after sorties during WWII. It has been used to to ask the question, "where would you increase the armor on this plane to improve survivability." The answer is, of course, everywhere there is NOT indicated battle damage. This is because it's ridiculous to assume the German Luftwaffe didn't shoot the planes everywhere they could, so the ones that returned with battle damage are the ones that were shot in places that were not vital. The ones shot in the blank areas did not come home.

All fine, up to this point. But, here's where we have to be careful.

Survivor bias is what happens to your data when the population you sample has been filtered by some process. Like the position of bullet holes is the data, the population are bombers, and the process was going on sorties over enemy territory in wartime.

So, in order for selection bias to apply, we should have data, the population the data has been gathered from, and the potential filtering process.

We have seen people assert conclusions - I have seen bless all of anyone even vaguely attempting to take data from any population.
 

Waller

Hero
My premise is that while D&D has always been big in the TTRPG marketplace, since 5E was released a decade ago it's seen double digit growth is an indication of a quality product. It's the best measurement we have.
Yes, we know. You’ve said it a LOT of times. I’m sure you’ll say it a lot more times.
 

Snarf Zagyg

Notorious Liquefactionist
This is another goalpost movement. Now its not about popularity but about sales growth? I thinks folks have demonstrated that low quality things are very popular and sell like hot cakes (see what I did there???). You need more than this to prove quality, instead you are just demonstrating it has value to a lot of people. They are not one and the same.

For the record, I am not saying 5E is low quality, I am saying that popularity and big sales don't make it high quality.

No. This is just as incorrect as the OP's point that people are trying to rebut. (And I don't mean to single you out, because a LOT of people are making the same exact mistake, although @Willie the Duck had a very good post while I was writing this).

Objective and subjective actually have ... meanings. And there is almost NOTHING that we discuss and argue about on these boards that is objectively measured. Depending on how you describe "popularity" (such as "sales in the last year" or "most people currently playing") you could be discussing popularity using objective metrics. Quality, on the other hand, is subjective unless you define it very narrowly ("Tesla cars have had the highest percentage of vehicles sold subject to serious recalls as defined by the United States government during the time period in question.")

This continues to be a source of frustration because we see the same argumentative loop occur. Now, these words can have different meanings in context (for example, "The judge struggled to be objective.") But when it comes to these types of qualities, it's pretty simple-

"Objective" can be measured using an outside referent that is agreed upon. "The temperature right now is 89 degrees." "This year, the company made more in revenue than last year." "That book is 30 pages longer than the other book." "That burger has 300 more calories than the other burger, and 10 more grams of fat." "This car has better gas mileage than the other car." "That SUV has more cup holders than the Ferrari." In short, comparisons that rely on agreed-upon outside facts.

"Subjective" doesn't. It's everything else.

Which is why two things keep occurring-

1. There really shouldn't be any debate about things that are "objective." The burger either has 300 more calories, or it doesn't. While there might be an error in measuring, once the measurement is established there is no real point in arguing about it.

2. Instead, all the arguments are about subjective ...qualities ... and/or the implications of those objective facts. But because people feel the need to assert that they are right, we continually see the jujitsu of people asserting that subjective qualities (like ... quality, or design, or how "good" something is) is not subjective, but is objective. Because if it's objective, then they can argue that they are objectively right (they're not).

Here, we see this repeat. Let's assume that the use of popular refers to the objective measures of "most people currently playing" and "most books sold" and "most revenue by a TTRPG game." By those three metrics, 5e is the most popular TTRPG game. I don't think that's really in dispute (in other words, I don't think most people would disagree that this is a fair inference to make from those facts).

That said, there is no objective measure for something being "good," or "well-designed," or "high quality," as those are subjective qualities. That's where the OP fails.

What I do think is correct is what I often say- popularity (high sales, dominating the market, etc.) is something that people should pay attention to. Because whether or not it's "good" or "bad" it does usually mean that there is some quality that it has that is appealing to large numbers of people.

Essentially, this leads to frustration to people who don't prefer what is popular, and prefer other things. I prefer a manual transmission in a car- for many reasons, that isn't popular as a giant SUV or pickup truck with an automatic and a lot of cupholders. There are people that prefer free jazz to Harry Styles. I have never watched a single episode of NCIS, but not only is it the top rated TV show ... I just looked at the highest-rated scripted shows on broadcast TV and realized I have never seen a single episode of any of the top 20 for 2021-22.

Some things that are really popular just aren't for me. And that's okay. But I also realize that people who are making things that are popular are designing things to appeal to people ... that aren't me. And instead of simply griping that the design is bad, I do make the attempt to understand why that design works for others, even if it's not something I prefer.

Which comes back around to 5e. The most common argument is not some anti-intellectual "It's popular, therefore it's good." Far more often we see people refuse to see that the things that they don't like might be contributors to the popularity of the product; in other words, they might hate all the cupholders or the pop stylings of Harry Styles, but other people seem to like them.

(As always, IMO, YMMV)
 

Maxperson

Morkus from Orkus
My premise is that while D&D has always been big in the TTRPG marketplace, since 5E was released a decade ago it's seen double digit growth is an indication of a quality product. It's the best measurement we have.
I wonder what happened about a decade ago. :unsure: Oh yeah! Fantasy games, RPGs and shows entered the main stream.

You're assuming that this growth is entirely due to 5e's quality, rather than the fact that D&D, and other traditionally non-main stream games entered the main stream at about that time. Had 2e or 3e been released 10 years ago, it would have seen a huge increase from that as well.
 

Oofta

Legend
The point of the picture was not about the quality of bombers, but, rather, as a fairly well-recognized illustration of survivorship bias. The argumentation that "if the majority of people using a product works for them then it is a quality product" is a form of survivorship bias, as the argument ignores those who have stopped using the product because it doesn't work for them. The argument centers around a form of selection bias. It only selects for the majority of those whom the product works for. To be clear, I don't think that a TTRPG needs to work for everyone, but I think we should also identify survivorship bias when we see it. A better set of questions may involve asking who D&D 5e was designed for and who are the individuals and groups who continue using it and why?


5e works for you. Why does it work for you whereas other games may not? What design qualities does 5e have that make it work for you or that you like? What does it empower you to do as a player or a game master? Does it deliver the experiences that it is designed for and how? How is the writing, production quality, and overall value of the product? I think that there are plenty of metrics you can talk about with 5e as a quality product without ever once needing to appeal to its popularity or growth.

I can accept that [quick google search later] a Ford Maverick pickup truck is considered high quality by people who like pickup trucks even though I will never buy one.

As far as why I like 5E? Many reasons. It holds together with minimal tweaking at higher levels. The rules are consistent and flexible while having very few things that I consider broken. It's easy to modify, combat it relatively quick and so on.

But I'm probably not the one to ask. I'm the old school core audience, although I was ready to give up on 4E at the end of it's life cycle I probably would have just gone back to 3.5 or Pathfinder. The people you should ask are the millions of people that started playing since 5E's release.
 



payn

Legend
No. This is just as incorrect as the OP's point that people are trying to rebut. (And I don't mean to single you out, because a LOT of people are making the same exact mistake, although @Willie the Duck had a very good post while I was writing this).

Objective and subjective actually have ... meanings. And there is almost NOTHING that we discuss and argue about on these boards that is objectively measured. Depending on how you describe "popularity" (such as "sales in the last year" or "most people currently playing") you could be discussing popularity using objective metrics. Quality, on the other hand, is subjective unless you define it very narrowly ("Tesla cars have had the highest percentage of vehicles sold subject to serious recalls as defined by the United States government during the time period in question.")

This continues to be a source of frustration because we see the same argumentative loop occur. Now, these words can have different meanings in context (for example, "The judge struggled to be objective.") But when it comes to these types of qualities, it's pretty simple-

"Objective" can be measured using an outside referent that is agreed upon. "The temperature right now is 89 degrees." "This year, the company made more in revenue than last year." "That book is 30 pages longer than the other book." "That burger has 300 more calories than the other burger, and 10 more grams of fat." "This car has better gas mileage than the other car." "That SUV has more cup holders than the Ferrari." In short, comparisons that rely on agreed-upon outside facts.

"Subjective" doesn't. It's everything else.

Which is why two things keep occurring-

1. There really shouldn't be any debate about things that are "objective." The burger either has 300 more calories, or it doesn't. While there might be an error in measuring, once the measurement is established there is no real point in arguing about it.

2. Instead, all the arguments are about subjective ...qualities ... and/or the implications of those objective facts. But because people feel the need to assert that they are right, we continually see the jujitsu of people asserting that subjective qualities (like ... quality, or design, or how "good" something is) is not subjective, but is objective. Because if it's objective, then they can argue that they are objectively right (they're not).

Here, we see this repeat. Let's assume that the use of popular refers to the objective measures of "most people currently playing" and "most books sold" and "most revenue by a TTRPG game." By those three metrics, 5e is the most popular TTRPG game. I don't think that's really in dispute (in other words, I don't think most people would disagree that this is a fair inference to make from those facts).

That said, there is no objective measure for something being "good," or "well-designed," or "high quality," as those are subjective qualities. That's where the OP fails.

What I do think is correct is what I often say- popularity (high sales, dominating the market, etc.) is something that people should pay attention to. Because whether or not it's "good" or "bad" it does usually mean that there is some quality that it has that is appealing to large numbers of people.

Essentially, this leads to frustration to people who don't prefer what is popular, and prefer other things. I prefer a manual transmission in a car- for many reasons, that isn't popular as a giant SUV or pickup truck with an automatic and a lot of cupholders. There are people that prefer free jazz to Harry Styles. I have never watched a single episode of NCIS, but not only is it the top rated TV show ... I just looked at the highest-rated scripted shows on broadcast TV and realized I have never seen a single episode of any of the top 20 for 2021-22.

Some things that are really popular just aren't for me. And that's okay. But I also realize that people who are making things that are popular are designing things to appeal to people ... that aren't me. And instead of simply griping that the design is bad, I do make the attempt to understand why that design works for others, even if it's not something I prefer.

Which comes back around to 5e. The most common argument is not some anti-intellectual "It's popular, therefore it's good." Far more often we see people refuse to see that the things that they don't like might be contributors to the popularity of the product; in other words, they might hate all the cupholders or the pop stylings of Harry Styles, but other people seem to like them.

(As always, IMO, YMMV)
Really, I dont see how this changes anything in my post.
 

Dungeon Delver's Guide

An Advertisement

Advertisement4

Top