Another insight I got while reading trough this thread:
I think a central concept here actually is delegation of control. For any game the primary source of control is the game rules made by the designer, as that outlines the written socially accepted basis for the activity you have agreed to participate in. For most board games these rules take extreme control over the activity, limiting player input to a discrete number of options.
Traditional roleplaying games with their rule 0 however do something quite drastic: It delegates a lot of the control game rules normally can exert to a single player instead. That means one player essentially can make statements that has the same binding effect as if it had been in a traditional rulebook.
RPGs that do not hand this kind of control to the DM generally claim that this is in order to empower the players, but indeed they rarely give any authority to anyone to "change the rules". In other words, when it come to the "game rule" level of privileges, the game designer retains all control.
I think this distinction might be quite important with regard to what kind of experiences might be possible with the two kinds of games. By being delegated rules levels of control in a traditional rpg, a GM can do certain things that is really hard to acheive with a written ruleset. Maybe most importantly might be dynamic delegation of power according to the state of the game.
‐--------------------------
Let me give an example from a D&D campaign:
The first session I ran a quite simple railroad, mainly to get the players to get a bit used to each other and their characters. I retained full control over narrative, while they filled in flavor.
Second session when it had become clear they had gotten a bit into the setting, and had given clear indication that they were more interested in following other aspects of the setting than I had originally envisioned I started the session by asking them to each describe one thing they wanted to happen in the session. My challenge as a GM was to see if I could manage to get all events in sensibly (I think I managed

). This way I delegated a lot of control over the session to the players in a structured way, allong with effectively creating a "minigame" for myself.
Soon after we had a session I allowed the player that had lost the previous session to narrate quite freely what he had been up to. This set the stage and theme for the rest of the session (and indeed much of the remaining campaign). Again delegating a lot of controll over the narative to that player.
Soon after one of the group got captured and interrogated, and I delegated full control over the interrogators to the other players. That is possibly the most memorable roleplaying moment I have ever experienced.
After quite a few sessions they had started to get a bit tired of all the chaos they were causing, and escaped it without any clear direction for what to do next. At that point I ran a pre made adventure, effectively delegating most of the narrative control to the adventure author.
------------------------
This was all with the same characters. Noone needed to learn new rules. Still the feel, style and power dynamics changed from session to session giving unique experiences. This is the kind of things I really struggle to see how to manage with non-traditional rpgs.
In particular I can easily see how I as a DM can encourage and create mini sub system that effectively delegates power to players and restrict myself in similar ways as in a non-traditional RPG, as I have rule-level powers. However I have struggled to see how I could use any of the non-traditional rpgs I know to run a premade adventure if I so would like. The problem is that those rules do not delegate the power to me that is needed to further delegate the required control to the adventure author.
So in conclusion:
The main reason I can see for "needing" the level of control D&D grant to DMs are to be able to further delegate that control.