D&D 5E What is Quality?

James Gasik

Legend
Supporter
So you're having a problem with a product. It's not doing what you need it to do. Perhaps you've encountered a design flaw or shortcoming that you feel attention should be brought to, so that you can find out if it's not just you- if enough other people have the same issues, maybe it's something that needs to be fixed.

There's always going to be other opinions, of course. People who are like "hey, look, it does all these other things, you should be happy with what you got", but humans are, by nature, never truly happy with anything for long. Which is one of the reasons why we innovate.

If people were happy with an initial product, we'd still be playing the original version of D&D, for example.

But comments like "it must be good, many people like it", "it must be good because it makes a lot of money" or "i have never had this problem" aren't super useful.

Because at the end of the day, you have two choices. Either find a product that does what you want it to do, or find a way to make it work. Or gripe at the product maker, but they likely won't do anything unless a lot of money is on the line.

Or, as Edward Norton says in Fight Club:

"A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall? Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X. If X is less than the cost of a recall, we don't do one."

A thing can be popular, make a lot of money, and have critical flaws. There's no reason to ignore the fact that it is flawed on the basis of it's success. Avengers: Endgame made a TON of money. However, you will find many people who point out that it's a flawed movie.

Quality =/= Entertainment value is a common dispute amongst fans and critics. One can be entertained, highly entertained, by something that, from an artistic standpoint, is garbage.

So while popularity and success are a value by which a thing can be judged, they are not the only values.

I'm not going to argue with 5e's success- the reason it's successful is the reason I engage with the game and attempt to play and run it. If it were not successful, I wouldn't need to. I might choose to anyways, if it was the ideal product for my needs, but that's neither here nor there.

Because it is popular, many people want to play it. For me, thinking about playing it or running it again, requires me to look at it critically, not turn my brain off and go "everyone likes it, so surely if I don't like something, I must be insane".

I could be insane, but I'd like a little more evidence than "most people don't agree with you".
 

log in or register to remove this ad

LadyElect

Explorer
The perception of “quality” is and will always be a subjectivist concept, whether applied to D&D or any number of analogues already presented here—fashion/accessories, food/beverages, music. This is why dialogues from an objectivist perspective never truly resolve. Popularity exists as an abstraction of shared sensibility and perspective. Due to the nature of its enumerability, many lean on it as the closest thing to an objective resolution, but it still fails when individually contested. Similarly, a divergence of ideals by niche markets or individuals with field expertise/seniority are just as subject to scrutiny despite the perception of authority they may carry for others.

All that philosophizing is just to say it’s inherently fruitless to seek a resolution to this discussion. I like to take part regardless, personally.

5E has largely succeeded due to its palatability. It’s mathematically simple as far as user demand is concerned. The designers have continued to lean into its aspects of individualist expression within an era that thrives on having outlets for that. Early adoption and opinion boosted its self-supporting marketability and the streaming era has been a further boon in that regard.

Those are examples that at least help to account for its success for the popularity-as-quality crowd. The desirability of those and other aspects being contestable might be why others deny it.
 

So you're having a problem with a product. It's not doing what you need it to do. Perhaps you've encountered a design flaw or shortcoming that you feel attention should be brought to, so that you can find out if it's not just you- if enough other people have the same issues, maybe it's something that needs to be fixed.

There's always going to be other opinions, of course. People who are like "hey, look, it does all these other things, you should be happy with what you got", but humans are, by nature, never truly happy with anything for long. Which is one of the reasons why we innovate.

If people were happy with an initial product, we'd still be playing the original version of D&D, for example.

But comments like "it must be good, many people like it", "it must be good because it makes a lot of money" or "i have never had this problem" aren't super useful.
Just to nitpick this point: it could just be (the theoretical) OP. Your table could actually be the only one who find the basic crafting system in 5e too complex or whatever.

Often I feel those posts are more about seeking validation, although sometimes the answer is "actually there's a rule for you case, it's just buried in the books." (I see that one a lot in the PF2 fora than 5e, but it happens.)

On the other other hand - if OP is the only one who doesn't like the element and it's not because OP missed something, then OP has a point and a totally valid complaint. But if they are the only ones, they should probably fix it themselves for their table rather than expect WotC to do it for them.
Because at the end of the day, you have two choices. Either find a product that does what you want it to do, or find a way to make it work. Or gripe at the product maker, but they likely won't do anything unless a lot of money is on the line.
Yeah, that.
 

Campbell

Relaxed Intensity
So how do you define quality of design? I gave my reasons, ongoing success is just one and it's only a measurement. I think you have to look at the goals of a game, what the developers were trying to do and how they accomplished their goals.

I take a look at things like how well a game accomplishes what says it sets out to do. I take a look at things like how compelling core play loops are, value for complexity, time to table, how reward systems reinforce play, etc. Is it a subjective aesthetic judgement? Of course it is, but so is popularity. That's just depending on someone else's judgement. There is no objective measure of quality when comes to television, film, games or any other form of media.

If 5e were less popular, but still fundamentally the same product, would it have a lower quality design? If so design quality is not worth actually speaking to.
 


Oofta

Legend
The argument is not about saying it isn't 'a quality product', it's about not liking it when someone bombs discussions about flaws and critique with 'It's popular, so shut up' in order to cut that discussion off.

I started this because of your response to my post (and similar tangents)

Modularity is, and always will be, on a scale. I think 5E is more modular than at least the last couple of editions. It obviously could have been more modular.

No matter how flexible or modular they made the game, some people wouldn't have liked it. But the proof is in the pudding, or in the case of the team's goal of making a popular game, in the sales.

P.S. I don't remember anyone denying that at some point early on in development someone made the claim. The relevance of a one person overpromising in an interview is what I question.

The point was that if additional modularity was demanded, it could have been added. Instead WOTC has let 3PPs provide it, like Morrus's Level Up. If there were enough demand, WOTC would be in that space. Since sales continue to grow the additional modularity people want simply isn't necessary for WOTC to meet their goals. That's not a criticism of wanting more modularity even if I don't personally think it's necessary. It's acknowledging the reality of business decisions.

The goal for D&D developers was to make a broadly popular game. It worked.
 

Oofta

Legend
I take a look at things like how well a game accomplishes what says it sets out to do. I take a look at things like how compelling core play loops are, value for complexity, time to table, how reward systems reinforce play, etc. Is it a subjective aesthetic judgement? Of course it is, but so is popularity. That's just depending on someone else's judgement. There is no objective measure of quality when comes to television, film, games or any other form of media.

If 5e were less popular, but still fundamentally the same product, would it have a lower quality design? If so design quality is not worth actually speaking to.

I would also say that for a game like D&D to have a decent amount of flexibility in order to satisfy the needs of the broad base.
 

A car that is inefficient, uses cheap materials, offers minimal to zero safety features, etc. is not, by any reasonable definition, a quality car. It may be popular, it may sell extremely well, it may have other virtues, but quality is not among them. And yet there have been several cars which meet that definition, including the first mass production car, the Model T Ford. By design, it was made to be cheap to make, and to turn a profit mostly via sales volume. It was a direct rejection of the way cars had been designed up to that point, where they were luxurious, comparatively safe, finely-engineered custom builds, sometimes literally hand-machined from the best materials available. The Model T was specifically intended NOT to be a high-quality car, but rather to be a high-utility car.

Something being popular does not, in any meaningful way, communicate that it is a superior product, neither in design, nor in execution, nor in materials. Popularity tells you that, in the context where the product was deployed, it was seen as a desirable purchase. Higher quality products are not at all guaranteed to be the bestseller, not even guaranteed to be in any particular position. Indeed, the bestseller is usually the cheapest product that doesn't fail too often nor too severely. Even that last one, severity, is up for debate if the catastrophic failures are rare enough. Consider the Pinto, very popular until the whole exploding gas tank case came along.

A thing may sell, it may in fact sell extremely well, and still be a mediocre product. A thing may have very low sales, may barely sell at all, and still be an excellent product. Popularity does not meaningfully sort for quality. Note that this goes both ways; I'm not inverting the fallacy either and saying that popularity indicates something is a bad product. I'm saying that there is no relationship, not even correlation, between sales volume and quality. If there were, we'd see a lot fewer crappy, crappy blockbuster video games that somehow still made bank.

In specific, this reasoning is a form of argumentum ad populum, a subtype of appeal to authority. It would be a fallacy to assert that this means popular things are bad or wrong. Instead, popular things are...popular. There may be many reasons why they are popular/sell well, with quality being just one, and frequently quality is not the most important concern. If we wish to make assertions about the quality of a sold good, we must ground them in the actual...you know...qualities of that good. For physical objects, the materials from which they are made, aka the "production value." For most things, the design and utility (since most products have SOME kind of utility value, even if minimal). Ease of use, interoperability, elegance, etc. Sales don't tell you anything about these things.
 

James Gasik

Legend
Supporter
Speaking to D&D specifically, no matter how popular it is, it's going to be replaced by a new edition eventually. Wizards of the Coast will want to sell us all new books, and once sales dip to X level, the stars align, the prophecy comes to pass, or whatever metric they use to prognosticate these things reaches a given value, they will unveil the next big thing. It could be 2 years from now, 5 years from now, or 10 years form now (I'm going to be cynical and assume sooner rather than later, but it's not like I have special knowledge on this topic).

When this occurs, should the new game sell more product, does that mean the new game is simply better? Does that mean 5e was less of a game because it didn't make as much money?

Do we take market trends or inflation into account? A rising population?

Would 5e, compared to this hypothetical edition be said to have "failed" because it didn't make as much money?

Obviously not. But in this grim future, will people who were perfectly happy with 5e and have problems with 6e, 25e, Super Advanced Dungeons AND Dragons ("ampersands are so last century!") Mega Edition, or whatever they call it be told "there is nothing wrong with the new D AND D because it's making more money"?
 

Umbran

Mod Squad
Staff member
So how else are we supposed to judge quality of a product other than how well it sells and meets or exceeds it's targets?

Well, there are a few things we can talk about that differentiate "quality" from "meeting financial targets".

The first we might address is the McDonald's Phenomenon. McDonald's food sells well. But, it isn't healthy to eat frequently, and its nutrition content per calorie isn't high. It has carefully tuned (and high) levels of salt, fats, and sugars to entice the human hindbrain, but it you are looking for flavor beyond that, it doesn't have much going for it.

McDonalds food does have qualities we can say are, in some way, positive - it is inexpensive, it is highly available, and it is pretty consistent across most of the USA.

So does the massive revenue of McDonald's indicate they make "quality food"? Or is it "food with some sellable qualities"?

Then, we can talk about quality in other aspects. We can talk about cars - cars that go fast, are super popular, that people drool over, and sell well for their price category, but spend lots of time in the shop for maintenance or repairs. How "quality" is that car, compared to, say, a really reliable, efficient, but not very sexy Prius? That depends on what matters to you.

I submit that saying a product is, or is not, a "quality game" is not particularly meaningful until you talk about what qualities matter to you. Otherwise, it is too general except as a very broad statement of acceptance or rejection.

As for whether sales are quality... they are not a direct indicator of any particular quality you may care about. But high sales don't happen for zero reason either - so there's an indirect indication of something there. For McDonalds, sales probably does indicate that it is inexpensive, quick and available, which may matter when you are on a long drive to a convention.

So, overall - whether you are for or against a game, talking about "Quality" in a general sense probably isn't meaningful enough to bother with.
 

Minigiant

Legend
Supporter
Just remember. 5e is the most popular edition but almost every fan of it knows the PHB ranger and its subclasses were designed poorly. Most can say that the game was mostly great and still point to that blemish. So something can be high quality and have flaws worth open criticism.

Heck the Hunter and Beastmaster STILL don't follow the current design of 5e just becase WOTC doesn't want to state that the subclasses are missing its bonus spells and errata the PHB.
 

Campbell

Relaxed Intensity
My number one example when it comes to a lack of a link between game design quality and popularity is Vampire - The Masquerade. Don't get me wrong. It had great product design - a compelling setting, great art with a phenomenal sense of attitude, reached into a new market. Great business stuff. Great setting design. Crap game design. Overly complex systems in areas of the game that run counter to its core ethos, terrible GMing procedures, reward systems that run counter to its procedures. The only way to get a functional game out of it was to pretty much ignore all of its rules.

I still have nightmares about Celerity Turns.
 

James Gasik

Legend
Supporter
My number one example when it comes to a lack of a link between game design quality and popularity is Vampire - The Masquerade. Don't get me wrong. It had great product design - a compelling setting, great art with a phenomenal sense of attitude, reached into a new market. Great business stuff. Great setting design. Crap game design. Overly complex systems in areas of the game that run counter to its core ethos, terrible GMing procedures, reward systems that run counter to its procedures. The only way to get a functional game out of it was to pretty much ignore all of its rules.
Boy as a long time VtM fan, I can't agree with this enough. Just thinking about Obtenebration level 3 and trying to make sense of it (let alone all the arguments about it) is the stuff of nightmares.
 

Vaalingrade

Legend
My number one example when it comes to a lack of a link between game design quality and popularity is Vampire - The Masquerade. Don't get me wrong. It had great product design - a compelling setting, great art with a phenomenal sense of attitude, reached into a new market. Great business stuff. Great setting design. Crap game design. Overly complex systems in areas of the game that run counter to its core ethos, terrible GMing procedures, reward systems that run counter to its procedures. The only way to get a functional game out of it was to pretty much ignore all of its rules.
To say nothing of the scenario design.

Anyone remember the NPC everyone hated, but who got more powerful and more prominent with every release until he was like a double werewolf mage-pire?

the 90's was filled with absolutely trash meta-arcs that sold gangbusters.
 

James Gasik

Legend
Supporter
To say nothing of the scenario design.

Anyone remember the NPC everyone hated, but who got more powerful and more prominent with every release until he was like a double werewolf mage-pire?

the 90's was filled with absolutely trash meta-arcs that sold gangbusters.
The lesson to be learned here is that crossover games are bad. Do not attempt!

Hey at least when Samuel Haight hit the Underworld, someone had the good sense to soulforge him into something useful!
 

My number one example when it comes to a lack of a link between game design quality and popularity is Vampire - The Masquerade. Don't get me wrong. It had great product design - a compelling setting, great art with a phenomenal sense of attitude, reached into a new market. Great business stuff. Great setting design. Crap game design. Overly complex systems in areas of the game that run counter to its core ethos, terrible GMing procedures, reward systems that run counter to its procedures. The only way to get a functional game out of it was to pretty much ignore all of its rules.
Similar arguments apply to PF1e. Its own designers were eventually forced to admit how flawed and broken its design was, even though, as Oofta and others are quite eager to point out, it managed to overtake 4e on one voluntary measure of physical sales. Like, the designers themselves were forced to admit that the game design of PF1e was holding them back because of its low quality, and that it thus needed to be replaced. A product that sold extremely well in its market...that its designers eventually called, pretty much straight-up, badly made.

To meet or beat your sales targets does not tell you that you made a high quality product. It means that the state of the market, the state of your company, your marketing efforts, the interests of the consumer, and the actual product itself aligned successfully. Only one of those things, the product itself, involves any notion of quality—which means you can (and, for many products, do) see situations where the other factors dominate the equation. E.g. Umbran's analysis of McDonald's. The food is precision-optimized to trigger pleasurable responses in the consumer, regardless of nutritional value or ingredient quality: precise levels of fats, salt, sugar, monosodium glutamate, etc. This, coupled with consumers finding utility in the food being inexpensive and quick, mean that it sells well, even though it is unhealthy, nutrition-poor, weak on flavor beyond those really basic hindbrain-pleasers, and low in variety. You know...the vast majority of reasons humans choose to eat food for apart from "not starving to death."
 

Umbran

Mod Squad
Staff member
Obviously not. But in this grim future, will people who were perfectly happy with 5e and have problems with 6e, 25e, Super Advanced Dungeons AND Dragons ("ampersands are so last century!") Mega Edition, or whatever they call it be told "there is nothing wrong with the new D AND D because it's making more money"?

They will be told that, just a much as they are told, that the game should absolutely be changed to change an individual person's pet preferences.

If you complain about sales being used as an indicator, you should also complain about "Well, I personally like this better, so your published game should match me!"
 

I don't care for McDonalds and haven't eaten there this century. On the other hand, McDonalds has extremely high quality control standards. You go into a McDonalds and order a Happy Meal and it will be a Happy Meal whether you're in Fairbanks or Miami. You'll also know that the place will be well lit and clean. On the other hand, while I don't eat out often I do enjoy a burger at Culver's now and then. For me? A burger and a shake on average has been more satisfying and enjoyable than highly rated expensive restaurants I've tried. So for me, by and large that burger is a higher quality. There's no accounting for taste. 🤷‍♂️ Just like most people can't tell the difference between an $18 bottle of wine and a $3,000 bottle of wine.
Having high quality control does not mean your product is of top quality. It guanrantees that it will be the same in all units of production. Nothing else. A fast food is not good for your health and while I do enjoy McDonald's, it does not mean that their food is top quality for my health. It is not.


So ... classical music is higher quality because you say so? I would say that they're different, it doesn't make one higher quality. It depends on the goals of the people making the music. For one person Wagner might be da bomb, for someone else it might be Rhianna. While Rhianna isn't on my Spotify list personally, she has very high quality production and concert standards. Many bands hit big and then flair out because they simply can't sustain a quality product. Whether that's because they were one-hit-wonders who just happened to get lucky or because of personal issues with the performers.
Not because I say so, but because most professionals in music will tell you so. Look at a partition of Bethoven, Mozart, Iron Maiden, Metalica and so on. You will see that partitions from hip hop, rap and pop are rather anemic both instrument wise and accord wise.

When it comes to 5E, if it wasn't a popular product they would have been like one-hit-wonder Blind Melon or Nickleback who were big for a while but had so, so many issues.
And so was Vampire the Masquerade. At a time it was bigger than D&D. A fad is just that, a fad.

If your books fell apart you should have sent them back for new ones. I did (twice, different books) and not only did they replace it with no questions asked and no receipt, they also sent me an additional product as well. I agree the printer of the books wasn't great quality but their customer service was quite high quality.
Learned that you could that a year after the repair were done. And my repair still hold fast! That was quality repair if you want my opinion ;)


Yeah, nothing is perfect. I'm not saying 5E is the highest quality RPG of all time because I don't know how you would measure that. But 5E seems to be a quality product for millions of people, more people play every year. For a game when there's so much competing for our spare time to accomplish what 5E has to me indicates that for a lot of people it is a quality product.
It sells. It is a testament for its popularity and I am really happy that it is so. But it does not mean that there are better RPG out there. IF D&D was such high quality, you would never see so many house rule and 3pp trying to correct its mistakes. The 5ed of D&D is a great one, no one dispute that. But it does not mean that it is really such high quality as an RPG.

People have voted with their dollars and time.
And that is what popularity amounts to. Nothing else, nothing more.

And again, I want to be clear. 5ed is so far one of the best edition of D&D. No argument there. But it is hardly perfect and some of its bad choices are reflected in the game play and the criticism we see on this very forum and many others. It is a workable edition that does not need that much tweak to work as you want. And in that, it is why it is a success.
 
Last edited:


Oofta

Legend
A car that is inefficient, uses cheap materials, offers minimal to zero safety features, etc. is not, by any reasonable definition, a quality car. It may be popular, it may sell extremely well, it may have other virtues, but quality is not among them. And yet there have been several cars which meet that definition, including the first mass production car, the Model T Ford. By design, it was made to be cheap to make, and to turn a profit mostly via sales volume. It was a direct rejection of the way cars had been designed up to that point, where they were luxurious, comparatively safe, finely-engineered custom builds, sometimes literally hand-machined from the best materials available. The Model T was specifically intended NOT to be a high-quality car, but rather to be a high-utility car.

Depends on how you define quality. I think that at the time, the Model T was a quality car because it was well suited for the bad roads at the time and quite versatile. Since it was not hand made (which IMHO is over-valued for discussions of quality) you could get replacement and add-on parts that were not available for any other vehicle. Even if other vehicles had been less expensive, it suited the needs of people better than other vehicles.

Any car from that era would not be considered high quality today, even if I do think some are awesome. I go back to my comparison of a Casio digital watch to a Rolex. A Rolex is far, far more expensive and decorative. It is not inherently a higher quality time keeping device. A Rolex is only high quality if you value the extravagant nature of the watch.

Something being popular does not, in any meaningful way, communicate that it is a superior product, neither in design, nor in execution, nor in materials. Popularity tells you that, in the context where the product was deployed, it was seen as a desirable purchase. Higher quality products are not at all guaranteed to be the bestseller, not even guaranteed to be in any particular position. Indeed, the bestseller is usually the cheapest product that doesn't fail too often nor too severely. Even that last one, severity, is up for debate if the catastrophic failures are rare enough. Consider the Pinto, very popular until the whole exploding gas tank case came along.

A thing may sell, it may in fact sell extremely well, and still be a mediocre product. A thing may have very low sales, may barely sell at all, and still be an excellent product. Popularity does not meaningfully sort for quality. Note that this goes both ways; I'm not inverting the fallacy either and saying that popularity indicates something is a bad product. I'm saying that there is no relationship, not even correlation, between sales volume and quality. If there were, we'd see a lot fewer crappy, crappy blockbuster video games that somehow still made bank.

In specific, this reasoning is a form of argumentum ad populum, a subtype of appeal to authority. It would be a fallacy to assert that this means popular things are bad or wrong. Instead, popular things are...popular. There may be many reasons why they are popular/sell well, with quality being just one, and frequently quality is not the most important concern. If we wish to make assertions about the quality of a sold good, we must ground them in the actual...you know...qualities of that good. For physical objects, the materials from which they are made, aka the "production value." For most things, the design and utility (since most products have SOME kind of utility value, even if minimal). Ease of use, interoperability, elegance, etc. Sales don't tell you anything about these things.

The RPG marketplace is competitive, it's easier than ever to publish a new game. I judge quality in part on if it delivers what people desire, for entertainment products like games. So yes, popularity is one of the best way of measuring whether or not a game is entertaining there is. If D&D had spiked and seen declining sales it would be a different story.

Do you have any actual ideas on how to judge quality?
 

Dungeon Delver's Guide

An Advertisement

Advertisement4

Top