Most of what you're saying only applies to certain editions of D&D and not others. Some were more focused, others less so. Some had rich and in-depth lore, others less so. There's very little, especially playstyles, that have a solid through line across D&D editions.
For me, that's the crux of it. 5E doesn't do anything particularly well as a game...unless let the players just win all the time without breaking a sweat is an intentional design choice for gameplay. 4E knew what it was trying to do and did it quite well. Not a lot of D&D players wanted that from a game called D&D, so it didn't last. 3X knew what it was trying to do and did it...until it fell under its own weight re: rules for everything. So too with 2E and AD&D. The rules produced a particular kind of play. Whether people liked that style or not largely determined whether they liked that edition or not.
No. D&D has been more or less focused depending on the edition. 5E absolutely tries to be all things to all people.
It all depends on what you mean by "benefit." Would the game be better designed as a game? Absolutely, 100%, without a doubt. Would sales go down? Absolutely, 100%, without a doubt. Sales would likely drop if they put out a more unified, directed, or focused vision of what they wanted the game to be. Anyone who didn't like that vision would leave the game.
Near as I can tell, WotC stumbled into 5E's popularity. They designed it as the last edition of D&D, as in this is it, the brand is being rested, one last chance to make the game. Only because of a quirk in the design, i.e. the encounters per day math and PC resources, WotC stumbled into a fantasy superhero game where the PCs are gods even from low levels, there's no risk or danger and the game delivers power fantasy by the truckload. But that's down to the adventuring day math and almost no one sticking to that, so PC resources wildly, dramatically, absurdly outstrip just about every single encounter put in front of them. And apparently that's exactly what D&D fans were in the mood for. Because...boom.
All that said, D&D is...generally speaking...fairly unified in its design goals. It's always been a monster-fighting game. It does that to varying degrees of success. But it's always about fighting monsters. That some people don't play it that way in no way changes that being the absolute fundamental core of the game's design across editions.