You are not incorrect, probably for the vast majority of D&D players, but I'm suspecting for many of us life-time DMs with long campaigns, the characters are not a minimal segment of the campaign. And this heads towards your second paragraph...
Sure, I've played and GMed plenty of D&D campaigns, and there's certainly a range when it comes to character focus. I don't know if the length of the campaign has anything to do with it as much as the focus of play and how that's determined. There are other factors at play, too, I expect, but I think that's central.
This is quite insightful. It is why we have these endless discussions, because of this variance of play that exists within D&D
I have to echo @Enrahim2's position in that D&D affords me as DM the opportunity to change the style of play from one session to the next and even from one scene to the next should I so wish. That ability to surprise my players - either through the storyline or by giving them more narrative control in situations or through the introduction of a mechanic from another game appeals to me creatively.
And I agree with your sentiment that
neither (system) is better than the other, except as they relate to our personal preferences, or those of others.
Well, it depends on which version of D&D. Some are far more specific than 5e in how they're meant to play and what procedures and techniques are to be used.
5e Itself allows for probably the broadest range of playstyles of all the versions of D&D. And for many, this is a feature not a bug. For me, it's increasingly a bug. I also think it is a bug for clear conversation.
This was all with the same characters. Noone needed to learn new rules. Still the feel, style and power dynamics changed from session to session giving unique experiences. This is the kind of things I really struggle to see how to manage with non-traditional rpgs.
In particular I can easily see how I as a DM can encourage and create mini sub system that effectively delegates power to players and restrict myself in similar ways as in a non-traditional RPG, as I have rule-level powers. However I have struggled to see how I could use any of the non-traditional rpgs I know to run a premade adventure if I so would like. The problem is that those rules do not delegate the power to me that is needed to further delegate the required control to the adventure author.
So in conclusion: The main reason I can see for "needing" the level of control D&D grant to DMs are to be able to further delegate that control.
I feel like if we define need in this way, where any and all authority comes from the GM, we're veering into "mother may I" territory.
If there are benefits to not having the GM as the sole authority... if there are ways for the game to be improved by allowing players to have more input... then those should be the expected mode of play. They shouldn't shift.
For example, on my turn in 5e, I should be able to move, take an action, and then also take a bonus action, if I have one that applies. My ability to do this should come from the rules, not the GM. If there is some reason I can't do one of those things on my turn, I should understand why according to the rules.
I think viewing "need" in the way you have here is meant to say that D&D can produce any kind of play, but other games can't do that... but I don't think that's necessary, nor is it true, and it really does push us into the mother may I caricature.
I'm still not sure I buy that the vast majority of D&D players are running through pre-published adventure paths. I'd love to see some information that actually backs up this assumption as my experience outside of my own game just doesn't gel with this assumption (and admittedly I could be wrong) but the vast majority of DM's I've interacted with are doing homebrew world and homebrew adventures or published campaign setting and homebrew adventures... sometimes mixing in a published adventure here or there.
I don't think anyone said anything about how the vast majority of folks play 5e. I would say a significant number of folks clearly do so, and I would point to the sales of the adventure books and the significant number of streamed games you can find on youtube and other places online where people are doing exactly that.
Additionally, I'd suggest that the adventures are a model for many people in how to run a game, and so they influence many folks who create their own settings and adventures.
The adventures are clearly a significant factor in how people play.