The designers said repeatedly during the playtest of 5e that magic items were supposed to be fully optional, something that many--including myself--balked at over the years. And people continued to insist that it was true, that the DM did not ever need to give out magic items ever, for any reason, no matter what, period. Now, again, perhaps that belief is mistaken. But it's more than a little disingenuous to argue that the text truly supports the idea that magic items were always intended to be present for most if not all groups, when the text explicitly says that isn't the case, as you yourself quoted with the screenshot of that page from Xanathar's.
Again: it is entirely possible that the developers are simply wrong to say that, or that the text contradicts itself, or that the statement is an Obi-Wan-style truth "from a certain point of view," etc. You can, quite easily, argue any of those things and probably more that I'm not considering. But it is explicit, in the text and from the designers' lips, that magic items were intended to be optional, without any explicit caveats or reservations. To pretend that the text literally, actually claims otherwise is to blatantly ignore the actual words on the page. Whether those actual words are correct is absolutely a subject of (furious!) debate, but they're there, and to pretend like they aren't is playing sillybuggers.
My understanding was that the developers were saying that magic items were not required to make the math of the game work. They were not trying to suggest in the DMG in 2014 that magic items as rewards were somehow a second class game mechanic.
That was not true of 4e, and it was not true of 3e. Both of those editions were built around the idea that magic items were 100% available and would scale with level and all characters had to get them. 4e only has your static level bonus be half your level, but the actual math of the system says your total bonus needs to be equal to your level. The enhancement bonuses from items and ability score bonuses from the level table were assumed to exist to make up the discrepancy. You found a +1 weapon, +1 armor, and +1 ring or cloak, and you found a better bonus item periodically, too. A 17th level character should have all +5 (3.x) or +3 (4e) items just for the games' monster math balance assumptions to work out. You needed those items just to reach par.
5e wasn't designed that way. That's a major change from the prior 15 years of design. In 5e, they built it so that, regardless of your level, a +1 sword made you better than you "should" be instead of simply putting you on par. Bounded accuracy basically forces it to work this way. That's all "magic items are not a required part of the design" means. It's a description of how they defined CRs, and what level of magic items they baked into the game. However, that's
not what people read. What people read was, "PCs don't need magic items as rewards at all anymore so you should not need to give any rewards out." It's not surprising that players and DMs did that, either, because the 5e 6-8 encounter rule lowballs encounter difficulty so much that it's hard to think that you actually need rewards. That design was originally an attempt to encourage players to satisfy the short rests that some classes require to reach parity with other classes, but as the edition progressed and the developers learned the many ways that doesn't actually work and have done tons of data collection, they have now mistaken their observed data of what they told people to do and now assume that it's how people actually want to play.
The trouble with that reading is that fighters have always been much more equipment dependent than wizards, and that really didn't change at all. If you look at the random treasure generation rules and extrapolate them, it's really clear that they expect you to quite regularly give out magic items as rewards (a lot of them!) because that's exactly what the tables do. They didn't require magic items, but the author of the DMG 100% still expected you to give out rewards... and that meant magic items!
The alternative, too, is that the entirety of Chapter 7 in the 5e DMG would be optional content that you're not supposed to use at all. Just compare the page count of the explicitly optional rules in the PHB (multiclass rules and feats) to the page count used by the magic items in the DMG. Or to the
non-magic item rewards in the DMG. If, in 2014, WotC was planning to deprecate magic items, they did a godawful job at writing a DMG that would do it. This would be like publishing sci-fi monsters in the monster manual. And not
monsters you can re-theme as sci-fi. Actual monsters for sci-fi-only campaigns in the monster manual. It just makes no sense to do it that way just from a raw publishing standpoint.
Even worse, deprecating magic items means that the new spellcaster limitation of concentration or the reduced effectiveness of spells really doesn't do anything to curb spellcasters like your original design goals intended. It can't when your martial classes now don't meanigfully scale
at all. Martials don't scale much at all except for hp now, and the low encounter difficulty masks the fact that the lack of rewards exacerbates the power level skew.
We can even see the changes. Historically, by the time Dungeon of the Mad Mage is released, I believe the module authors have been told something different: don't put magic items into the module if they will duplicate the "basic" items given out by
Adventurer's League v8.2 treasure checkpoints.
But the module didn't say that to the normal DM. The game just assumes the DM magically knows to add in rewards. That's how you get a 25 level megadungeon with basically
no martial rewards of any kind at all, and virtually nothing else except potions, scrolls, and a very small number of other items. And now DMs pick up published modules and run entire campaigns without any item rewards because
that's what they think the game has told them to do.
Now you look at
the Adventurer's League v11.0 rules now they don't even include the treasure checkpoints or "basic" items. All they have is a single item if you start play at 5th level. This might be an improvement because it's not making two divergent module reward systems and then not telling DMs about it, but it's still clear that they have completely altered the way that rewards are given out from how they actually designed the game in 2014 and operated it in 2018.
I think it was very much
not the intent and
not the original design to just have no player rewards in the entire game. That goes against D&D's pulp fantasy roots and leans
very heavily into heroic or epic fantasy. I don't really think that's a great design choice for a game that they
still claim is all things to all players. I think their own communication has poisoned the well. I think their own choices to almost exclusively produce modules that have epic storylines has contributed as well. And I think they've just kind of gone with it because it's easier to not have to plan around magic items.
