This is the thing, though: What is a 'better result?' Because people will not agree on that.
As there is usually only one DM at that table, obviously, the DM is the judge of 'better.'
Another example is the DM who applies very specific real-world physics to non-magic users, but lets magic do everything...discouraging non-casters.
That's his choice, of course. If a DM wants his campaign would to emphasize the importance or supernatural aspect or primacy of magic, that would be one way to do it.
So ultimately, what my toon does, succeeds or fails, isn't up to me? It's just GM fiat? Why play?
Ultimately, what your character chooses to do (attempt) is up to you. What he is able to do, and whether any instance of doing them succeeds or fails is up to the DM, yes.
Arbitrary GM fiat and arbitrary mechanistic application of the rules are not the only alternatives, and even if they were, a GM could use a mix of the two.
The sense of accomplishment in overcoming the challenge is huge because the challenge doesn't calibrate itself to your capability. The lock doesn't care if you're a level 1 rogue or a level 20 rogue.
This is nonsense. The DM determines how difficult things are. He can take the party's abilities into account to make the game challenging ('tailored') or he can create the world independently and let the players try to figure out which challenges they should take on and in what order ('status quo'). That's how it is in 5e, that's how it was in every prior edition, that's how it is, period.
I think this is one of the philosophical differences. If a game wants you to succeed, it's not offering a very meaningful choice - either way, you're probably going to succeed. Either way, you win. Either way, the good guys emerge victorious. The fight vs. the bridge doesn't actually affect your chances of the mission succeeding or failing much.
It's a genre trope, as well. In the broader fantasy genre, and in heroic stories of any kind, really, heroes face many challenges, most of which seem to range from risky to overwhelmingly dangerous, with the Hero surviving by the skin of his teeth repeatedly. Design a game with random resolution to match that appearance, and every PC will die before any adventure is completed. No one will ever reach 2nd level. Rather, the game has to tune it to the rate at which heroes survive, not the way their challenges are portrayed. And the hero generally lives through all the challenges that take him up to that final confrontation at the climax of the story...
5e throws back to pre-4e game-centered philosophy in that in general it is perfectly okay with you failing disastrously, if that's how it plays out.
It's not like a game could actually get upset with you for failing, but I think I get what you're trying to say. In the early days of RPGs, they were still very much like wargames, they set up a scenario, and did your best to achieve victory conditions with the units provided. The challenge was to achieve victory, and there were no particular constraints on how. As RPGs - and gamers - got more sophisticated, they started thinking in other terms, genre, character concepts, and stories, and the 'victory conditions' shifted. If you're playing a game that seeks to emulate a genre where characters are killed off only rarely, and even then in meaningful ways, a TPK is a complete failure. Not just a mere loss for one player, but a failure of the DM & all players involved.
D&D, eventually (far behind the curve, as always) introduced some improved mechanics and guidelines to help the DM do his part to run successful games. The resulting CR guidelines often failed - producing a 'speedbump' fight that turned into a TPK, or a tough, climactic battle that turned into a rollover - but they were improved over the 15 years that D&D struggled to deliver on the idea. 5e's CR guidelines are back to being less than dependable, probably as an alternative to just chucking them entirely, in keeping with the classic D&D, wargame-like, philosophy.
I just look at it this way, for my purposes I don't need statements telling me that I can 'do it my way'. I've been DMing for pretty close to 40 years now, I do it my way.
Consider, though, that those statements are not hidden in the DMG somewhere, 'behind the curtain,' but are up-front in the most basic 'how to play' explanation of the game that every player should read.
If you have a stable group, that may not matter, but, if as I do, you run 5e at public venues (and, that's actually the only context in which I run 5e), it's very helpful to have everyone on the same page when it comes to the function, role & responsibilities of the DM. What 5e does, in that sense (that classic D&D also did, and which 3.0 tried to do unsuccessfully) is to shape the community consensus on the issue - that a big part of its "DM Empowerment" agenda.
The role, for me, for rules is to provide RULES, things I can just read and say "ah, OK, so when X happens the player makes a check, like so." Now, if I don't like the way something works, then I change it. They work OK in 5e too, most of the time, but they seem to have this fetish with writing incomplete or niche rules sometimes.
Nod. They leave rules open to interpretation, even fairly basic rules that will see frequent use. That way the players become accustomed to the idea of the DM making rulings that need to be respected being necessary just to play the game. That they also help the DM shape the game experience - impose class balance, present challenges, tell a story, assure the players are having fun with it, and so forth - may not be so evident, but are supported by that acclimation to DM is sole arbiter.
Again, I don't know what would lead anyone to believe that 4e is more limited in this respect than 5e.
Well, there was that 7-year campaign of lies and misinformation known as the edition war.
The issue here is its a lot harder to ADD range to a compressed system. Its not that hard to trim range out of a wider system because you can always just ignore part of the range in various ways.
True. You can snip out a sub-set of the range, you can tune adventures around a narrower bound about the point on that range the PC are currently at.
Adding range to something like 5e would be a matter of adding levels. Currently, like 3.x, it's 20 levels. You could pile Epic levels on top of that, like 3.x did.
My concern, as a GM, is with managing pacing (including the contributions to pacing of successes and failures) in a way that does not unduly favour one player over another. I don't feel that 5e's guidelines help me a lot with that, because (whether or not I treat the DCs as "world set") they don't give me advice on how those DCs relate to expected PC capabilities and player resources.
How could they? Party resources are going to vary with party composition. Some parties may be heavy with short-rest-recharge resources, other with long-rest-recharge. Some will have more hps and more short-rest healing, others more in-combat healing available. There's no way to present much of an idea of what's 'expected.'
The DM in 5e is just a more responsible role than it was in the prior two editions. More depends on the DM's talents, experience, system mastery & artistry.
(I also think you have mischaracterised Gygax's advice in his DMG. He is quite emphatic that overriding the dice in respect of action resolution would be contrary to the most important tenets of the game. The only bit of action resolution override he countenances is in how to adjudicate a PC being dropped to zero hit points. But that's something of a tangent.)
He might not have agreed with taking it as far into the moment-by-moment resolution system as 5e has, but the basic idea that the rules are a 'starting point,' that the DM works from, rather than something he must abide by has always been there. EGG might have preferred the DM actually write down any variations he had in mind, for instance, rather than making them up on the fly. Then again, he also counseled DMs to keep ahead of their players in terms of rules-knowledge. So, whether you have some variant in Gygaxian D&D (and don't volunteer the details of it to the players) or are ruling arbitrarily in 5e, you're an Empowered DM, and the player experience is comparable: one of mystery and discovery.