D&D 5E What is Quality?

[FrogReaver]

Popularity alone is often based on the intrinsic qualities. One of the things that does contribute to popularity, though, is brand name recognition/loyalty, which is something that D&D has in spades. Of course, that has nothing to do with the actual quality of the game. It applied to 2e, 3e, 4e and 5e.

Fully agree, but I do think the basic product needs to meet some minimum quality threshold or external factors like brand name recognition aren’t enough to maintain popularity.

Not saying that minimal level would be considered ‘good quality’. Maybe it’s ‘not terrible quality’.

 

[Aldarc]

Going by the implied argument of the last example, I don't think either of these are really that apt of comparisons to the topic of RPGs. Leaving aside the watch/clock-as-jewelry/art argument, the primary function of a watch is very obviously to tell time (and something like losing minutes a close-to-inarguable deficit in that regard). The bomber has a few competing functions (flying in general, carrying bombs, accurately dropping bombs, continuing to fly while being shot-up), but in general the quality is again fairly hard to argue against (even if you can argue about relative value of the above qualities, or even potentially how to measure). For TTRPGs, the comparable measure is (IMO) -- 'can the ruleset be used by someone to play a roleplaying session?' With few exceptions -- Hybrid being maybe an RPG at all but maybe just someone's word salad that includes some RPG framing, and FASA's 1980s Master of the Universe rpg (where there are rules referenced which never actually show up in the rulebooks) -- all TTRPGs meet that standard and most of the qualities used in arguments about which ones meet the standard better than others not being objective or even semi-universally agreed-upon.

All this leads me back to my overall conclusion that RPGs are closer to movies or music than burgers or clocks -- they all do the basic necessary requirements to qualify as the thing, some but not all of the components of their quality can be measured unambiguously, and there is no consensus on comparing product A with strengths in X and Y to product B with strengths in Z and W.

The point of the picture was not about the quality of bombers, but, rather, as a fairly well-recognized illustration of survivorship bias. The argumentation that "if the majority of people using a product works for them then it is a quality product" is a form of survivorship bias, as the argument ignores those who have stopped using the product because it doesn't work for them. The argument centers around a form of selection bias. It only selects for the majority of those whom the product works for. To be clear, I don't think that a TTRPG needs to work for everyone, but I think we should also identify survivorship bias when we see it. A better set of questions may involve asking who D&D 5e was designed for and who are the individuals and groups who continue using it and why?

I probably should have phrased this whole thread differently. In retrospect it should have been "Can you determine quality of a TTRPG?"

5e works for you. Why does it work for you whereas other games may not? What design qualities does 5e have that make it work for you or that you like? What does it empower you to do as a player or a game master? Does it deliver the experiences that it is designed for and how? How is the writing, production quality, and overall value of the product? I think that there are plenty of metrics you can talk about with 5e as a quality product without ever once needing to appeal to its popularity or growth.

 

[Umbran]

This is a graphic created showing the distribution of battle damage to the B-25s that returned to base after sorties during WWII. It has been used to to ask the question, "where would you increase the armor on this plane to improve survivability." The answer is, of course, everywhere there is NOT indicated battle damage. This is because it's ridiculous to assume the German Luftwaffe didn't shoot the planes everywhere they could, so the ones that returned with battle damage are the ones that were shot in places that were not vital. The ones shot in the blank areas did not come home.

All fine, up to this point. But, here's where we have to be careful.

Survivor bias is what happens to your data when the population you sample has been filtered by some process. Like the position of bullet holes is the data, the population are bombers, and the process was going on sorties over enemy territory in wartime.

So, in order for selection bias to apply, we should have data, the population the data has been gathered from, and the potential filtering process.

We have seen people assert conclusions - I have seen bless all of anyone even vaguely attempting to take data from any population.

 

[Waller]

My premise is that while D&D has always been big in the TTRPG marketplace, since 5E was released a decade ago it's seen double digit growth is an indication of a quality product. It's the best measurement we have.

Yes, we know. You’ve said it a LOT of times. I’m sure you’ll say it a lot more times.

 

[Snarf Zagyg]

This is another goalpost movement. Now its not about popularity but about sales growth? I thinks folks have demonstrated that low quality things are very popular and sell like hot cakes (see what I did there???). You need more than this to prove quality, instead you are just demonstrating it has value to a lot of people. They are not one and the same.

For the record, I am not saying 5E is low quality, I am saying that popularity and big sales don't make it high quality.

No. This is just as incorrect as the OP's point that people are trying to rebut. (And I don't mean to single you out, because a LOT of people are making the same exact mistake, although @Willie the Duck had a very good post while I was writing this).

Objective and subjective actually have ... meanings. And there is almost NOTHING that we discuss and argue about on these boards that is objectively measured. Depending on how you describe "popularity" (such as "sales in the last year" or "most people currently playing") you could be discussing popularity using objective metrics. Quality, on the other hand, is subjective unless you define it very narrowly ("Tesla cars have had the highest percentage of vehicles sold subject to serious recalls as defined by the United States government during the time period in question.")

This continues to be a source of frustration because we see the same argumentative loop occur. Now, these words can have different meanings in context (for example, "The judge struggled to be objective.") But when it comes to these types of qualities, it's pretty simple-

"Objective" can be measured using an outside referent that is agreed upon. "The temperature right now is 89 degrees." "This year, the company made more in revenue than last year." "That book is 30 pages longer than the other book." "That burger has 300 more calories than the other burger, and 10 more grams of fat." "This car has better gas mileage than the other car." "That SUV has more cup holders than the Ferrari." In short, comparisons that rely on agreed-upon outside facts.

"Subjective" doesn't. It's everything else.

Which is why two things keep occurring-

1. There really shouldn't be any debate about things that are "objective." The burger either has 300 more calories, or it doesn't. While there might be an error in measuring, once the measurement is established there is no real point in arguing about it.

2. Instead, all the arguments are about subjective ...qualities ... and/or the implications of those objective facts. But because people feel the need to assert that they are right, we continually see the jujitsu of people asserting that subjective qualities (like ... quality, or design, or how "good" something is) is not subjective, but is objective. Because if it's objective, then they can argue that they are objectively right (they're not).

Here, we see this repeat. Let's assume that the use of popular refers to the objective measures of "most people currently playing" and "most books sold" and "most revenue by a TTRPG game." By those three metrics, 5e is the most popular TTRPG game. I don't think that's really in dispute (in other words, I don't think most people would disagree that this is a fair inference to make from those facts).

That said, there is no objective measure for something being "good," or "well-designed," or "high quality," as those are subjective qualities. That's where the OP fails.

What I do think is correct is what I often say- popularity (high sales, dominating the market, etc.) is something that people should pay attention to. Because whether or not it's "good" or "bad" it does usually mean that there is some quality that it has that is appealing to large numbers of people.

Essentially, this leads to frustration to people who don't prefer what is popular, and prefer other things. I prefer a manual transmission in a car- for many reasons, that isn't popular as a giant SUV or pickup truck with an automatic and a lot of cupholders. There are people that prefer free jazz to Harry Styles. I have never watched a single episode of NCIS, but not only is it the top rated TV show ... I just looked at the highest-rated scripted shows on broadcast TV and realized I have never seen a single episode of any of the top 20 for 2021-22.

Some things that are really popular just aren't for me. And that's okay. But I also realize that people who are making things that are popular are designing things to appeal to people ... that aren't me. And instead of simply griping that the design is bad, I do make the attempt to understand why that design works for others, even if it's not something I prefer.

Which comes back around to 5e. The most common argument is not some anti-intellectual "It's popular, therefore it's good." Far more often we see people refuse to see that the things that they don't like might be contributors to the popularity of the product; in other words, they might hate all the cupholders or the pop stylings of Harry Styles, but other people seem to like them.

(As always, IMO, YMMV)

 

[Maxperson]

My premise is that while D&D has always been big in the TTRPG marketplace, since 5E was released a decade ago it's seen double digit growth is an indication of a quality product. It's the best measurement we have.

I wonder what happened about a decade ago. Oh yeah! Fantasy games, RPGs and shows entered the main stream.

You're assuming that this growth is entirely due to 5e's quality, rather than the fact that D&D, and other traditionally non-main stream games entered the main stream at about that time. Had 2e or 3e been released 10 years ago, it would have seen a huge increase from that as well.

 

[Oofta]

The point of the picture was not about the quality of bombers, but, rather, as a fairly well-recognized illustration of survivorship bias. The argumentation that "if the majority of people using a product works for them then it is a quality product" is a form of survivorship bias, as the argument ignores those who have stopped using the product because it doesn't work for them. The argument centers around a form of selection bias. It only selects for the majority of those whom the product works for. To be clear, I don't think that a TTRPG needs to work for everyone, but I think we should also identify survivorship bias when we see it. A better set of questions may involve asking who D&D 5e was designed for and who are the individuals and groups who continue using it and why?


5e works for you. Why does it work for you whereas other games may not? What design qualities does 5e have that make it work for you or that you like? What does it empower you to do as a player or a game master? Does it deliver the experiences that it is designed for and how? How is the writing, production quality, and overall value of the product? I think that there are plenty of metrics you can talk about with 5e as a quality product without ever once needing to appeal to its popularity or growth.

I can accept that [quick google search later] a Ford Maverick pickup truck is considered high quality by people who like pickup trucks even though I will never buy one.

As far as why I like 5E? Many reasons. It holds together with minimal tweaking at higher levels. The rules are consistent and flexible while having very few things that I consider broken. It's easy to modify, combat it relatively quick and so on.

But I'm probably not the one to ask. I'm the old school core audience, although I was ready to give up on 4E at the end of it's life cycle I probably would have just gone back to 3.5 or Pathfinder. The people you should ask are the millions of people that started playing since 5E's release.

 

[LadyElect]

There are people that prefer free jazz to Harry Styles.

 

[Oofta]

Yes, we know. You’ve said it a LOT of times. I’m sure you’ll say it a lot more times.

I was just told that increased growth was changing the goalposts.

 

[payn]

No. This is just as incorrect as the OP's point that people are trying to rebut. (And I don't mean to single you out, because a LOT of people are making the same exact mistake, although @Willie the Duck had a very good post while I was writing this).

Objective and subjective actually have ... meanings. And there is almost NOTHING that we discuss and argue about on these boards that is objectively measured. Depending on how you describe "popularity" (such as "sales in the last year" or "most people currently playing") you could be discussing popularity using objective metrics. Quality, on the other hand, is subjective unless you define it very narrowly ("Tesla cars have had the highest percentage of vehicles sold subject to serious recalls as defined by the United States government during the time period in question.")

This continues to be a source of frustration because we see the same argumentative loop occur. Now, these words can have different meanings in context (for example, "The judge struggled to be objective.") But when it comes to these types of qualities, it's pretty simple-

"Objective" can be measured using an outside referent that is agreed upon. "The temperature right now is 89 degrees." "This year, the company made more in revenue than last year." "That book is 30 pages longer than the other book." "That burger has 300 more calories than the other burger, and 10 more grams of fat." "This car has better gas mileage than the other car." "That SUV has more cup holders than the Ferrari." In short, comparisons that rely on agreed-upon outside facts.

"Subjective" doesn't. It's everything else.

Which is why two things keep occurring-

1. There really shouldn't be any debate about things that are "objective." The burger either has 300 more calories, or it doesn't. While there might be an error in measuring, once the measurement is established there is no real point in arguing about it.

2. Instead, all the arguments are about subjective ...qualities ... and/or the implications of those objective facts. But because people feel the need to assert that they are right, we continually see the jujitsu of people asserting that subjective qualities (like ... quality, or design, or how "good" something is) is not subjective, but is objective. Because if it's objective, then they can argue that they are objectively right (they're not).

Here, we see this repeat. Let's assume that the use of popular refers to the objective measures of "most people currently playing" and "most books sold" and "most revenue by a TTRPG game." By those three metrics, 5e is the most popular TTRPG game. I don't think that's really in dispute (in other words, I don't think most people would disagree that this is a fair inference to make from those facts).

That said, there is no objective measure for something being "good," or "well-designed," or "high quality," as those are subjective qualities. That's where the OP fails.

What I do think is correct is what I often say- popularity (high sales, dominating the market, etc.) is something that people should pay attention to. Because whether or not it's "good" or "bad" it does usually mean that there is some quality that it has that is appealing to large numbers of people.

Essentially, this leads to frustration to people who don't prefer what is popular, and prefer other things. I prefer a manual transmission in a car- for many reasons, that isn't popular as a giant SUV or pickup truck with an automatic and a lot of cupholders. There are people that prefer free jazz to Harry Styles. I have never watched a single episode of NCIS, but not only is it the top rated TV show ... I just looked at the highest-rated scripted shows on broadcast TV and realized I have never seen a single episode of any of the top 20 for 2021-22.

Some things that are really popular just aren't for me. And that's okay. But I also realize that people who are making things that are popular are designing things to appeal to people ... that aren't me. And instead of simply griping that the design is bad, I do make the attempt to understand why that design works for others, even if it's not something I prefer.

Which comes back around to 5e. The most common argument is not some anti-intellectual "It's popular, therefore it's good." Far more often we see people refuse to see that the things that they don't like might be contributors to the popularity of the product; in other words, they might hate all the cupholders or the pop stylings of Harry Styles, but other people seem to like them.

(As always, IMO, YMMV)

Really, I dont see how this changes anything in my post.