No. I'm saying he has final authority because there is literally nothing anyone can do to override him.
What they imagine is irrelevant. If, and I would never do this since I view the player as the one who controls PC actions, I said that your PC picks his nose. That's what happens whether you and the other players imagine it or not. You can either accept it or leave the game. Should you choose to leave it, your PC would become an NPC and the next group that encountered the NPC would encounter one who picked his nose.
The arguments made seem to partly rest on assumptions about what counts as "the game". So far as I can make out, you're picturing something like this
DM == Game
So that players can enter or leave the game that is virtually embodied in the DM. And seeing as, unlike a Monopoly set, DM is a free person and not a slave, they cannot be forced to change the game that they embody. The authority you refer to is just the basic autonomy of a person in a society that has the freedom to play games (setting aside arguments and variations on what that autonomy could amount to as being irrelevant here, to focus on the nature of games as a voluntary activity.)
Supposing the DM were a slave, then whoever owned that slave might be able to force them to change the game that they embody: making their supposed game playing an involuntary activity. An AI DM might occupy this sort of position. One might feel in that case that DM were not "playing" the game, only serving as an enabler of it. To an extent, it is this aspect of DM as enabler of the game that justifies the sort of model I've attributed to you. The contention is that seeing as DM is enabler, abandoning DM is like abandoning the Monopoly set: the game that they enable can't be played. Moreover, while Monopoly sets are duplicates, one can even make an argument that "the game" enabled by a given DM has only one copy.
Another model is something like this
DM ==> Game <== Players
I've added arrows to suggest that here "the game" exists as a function of both DM and players. Remove a player, and in this model a different game is played (following the same arguments I made above for DM.) No one participant has total authority over "the game". Commonly there are asymmetries in who has the right to say what, but in this model such submissions to "the game" require agreement (which is often tacit, based on prior agreements) because until everyone accepts it "the game" is ambiguous... or splinters. (There's always some fuzziness: I'm referring to instances here that can disrupt what participants say and will agree to next.) The pattern of agreement is most at issue when a participant ought to say something that another participant will find unwelcome (such that their agreement will be fraught.)
If right, then the gap between views may in part be explained by different models. The first model presents a tautology. "The game" is defined in terms of DM and it is meaningless to suggest that they don't have total authority over it. As many intuit, one can only go on to speak about some other game. The second model defines "the game" in terms of DM + players. "The game" exists between them, and to remove a player is once more only to go on to speak about some other game. It is as right in the second model to deny DM has total authority, as it is in the first model to assert it.
There is a resilence that ought to be acknowledged here: "the game" in common can go on to have life beyond the removal of one participant, provided a corpus carries it forwards. (It can be recognisably the same game even after the departure of one participant.) I suspect it's asymmetries in how "the game in common" is impacted by the departure of different participants that prompt folk to entertain the first model. Contrast perhaps Adventurers League play with individual home campaigns.