Bedrockgames
I post in the voice of Christopher Walken
Always IME, the problem I encountered with Gms explicitly self proclaiming Keepers of Reality/Causality/Plausibility (which nonetheless implicitly they are and should be, in classic Gm driven style) is that they tend to the extremes in enforcing the proclaim:
either by running a too strict railroad kind of game, in which the drama, the turning points are already established during prep, not accepting off the rails Pcs' declarations and course of action statements,
or, on the other side,
by a sandboxy style game where, though, nothing interesting really happens, and when one asks for "stuff" to happen, or Npcs to actively interact, they just dismiss it as naive requests of drama in their world of pure immersion and realistic (read boring) display of setting.
Interesting the fact that both behaviours don't like nor allow backgrounds for Player Chararcters, dismissing them as burdens soon to be relieved of as the game/story begins to unfold.
My opinion is that a component of apprehension, almost fear, is preventing a more collaborative and enjoyable playstyle; a deep concern about sharing some narrative aspects of Rpg, about listening to inputs from the table and adding those to the usual output of being a Gm, so to enrich the experience.
Again, IME in actual play: nothing to do with the ongoing diatribe in the OP, which I find interesting to read, btw.
Anyway I understand that in a long campaign, with players coming&going, the Gm is the Keeper of Continuity, and rightfully so; I'm ranting about the rigid, fringe behaviours that role, more often than not, leads to.
This is a post I would like to respond to because you raise concrete things about table play that I can wrap my mind around. This is much better in my opinion than a discussion about how like or unlike the real world a game setting is or can be.
I think you raise interesting points. I will give my take based on what I have seen at the table and in online discussion. I think the extreme cases of this do exist, just like they can in any style, and any GM who is overly rigid about play style is eventually going to run into tension with the right player or group. However, I think one thing that can often feed this extreme adherence to playstyle, where things that once may have been totally permissible within it are now forbidden because of ideas that have taken root as guiding principles or pillars, is online discussion and gaming theorizing itself. I've seen it on the immersive end and the sandbox end, where, because our style frequently clashes with more narrative styles online, as discussion unfolds, we tend to define ourselves against them. Which almost leads to an inverse GNS theory like mindset at times. Personally I am more in favor of rolling back the clock before these discussions, and simply asking whether we've thrown the baby out with the bathwater at various points when it comes to adventure structures, tools, and approaches to GMing. I also always try to keep in mind, there is more than one way to run a successful campaign or game session, and at the end of the day, anything you embrace has to pay dividends towards that end at the table. Sometimes you hit on an idea, and it works great for a year, two or three, but something in the group changes, you change, etc and what was working, stops working. So my only real guiding principle here is to do what I can to continue having successful play at the table.
That said, I think critiquing a play style by its extremes isn't terribly productive. It can be useful to caution against the more extreme ends, or at least be aware that not everyone is going to have fun at those extreme ends. But I run a lot of sandbox campaigns where the GM is the one who essentially plays the setting, and they just don't look like what you are describing. I am not averse to incorporating emulative genre elements, and I also don't mind working with player character background. However I do tend to draw more of a line there. I'll happily take suggestions from players and I may let them in whole or recommend changes so they fit the setting. The reason I do this isn't because of some blind adherence to "The GM is God", it is because I was miserable as a GM and Player in the 2000s 3E era when it just became normal to let players allow all of their background, character concepts, etc into gaming (at least in many of the groups I played with). I am absolutely happy if a player says something that fits and is cool. I will allow. I will even allow powerful backgrounds. For example the other day I had a player ask to be the prefect of the region the adventure was taking place. This was well outside the normal allowance of the system (players in the game I am running can start as rank 9B officials and prefects are much higher than that). But I understood the player's ability to run such a character, and knew it would be used to make things more interesting and add layers to the campaign, so I allowed it. And it worked great. He elaborated on some of the details and those were all permitted into the campaign. The thing I think is important to retain here is the GM needs to be able to say no. Whether it is because of power concerns, or because of things not jiving well with the setting or campaign (these latter two are particularly important to me), I think that is a key function that, at least for how I like to play, I don't want to relinquish (or see relinquished when I am a player). Again, I just want to emphasize, I am talking about my bread and butter, weekly campaign preference. I am totally open to other possibilities when we are not trying to keep our long term groups intact.
Just one other example here that may shed light on why some players actually like to feel like the world is external to their character. Again I really do have to advise against rigid adherence to this. There will always be things like edge case mechanics that deviate from this concept, but are not so overwhelming, so they add to the game with their presence. I have been play testing a new system. And the players in one of the groups are people I've gamed with for about two years. They are pretty mixed in terms of preferences. They seem to like drama. They are not afraid of words like plot or story. And so I made a new ability in the game (largely because of their tastes) that was called something like "Master Schemer". It allowed characters after the fact to make a roll and declare they had dome something devilishly deceptive like poison the very wine the NPC is drinking. This was meant to emulate something that comes up a lot in wuxia. I was very surprised by the strong negative and cautious reaction it received across several groups, but even among the more story oriented group. The reasons for disliking it did vary, but one of them was the players were being given control of what happened in the setting without having to walk through the steps of doing it. I thought it worked great from a story point of view, but because I think it stuck out because it didn't jive with the style we had developed together as a group (even though preferences were all over the map).
Just to defend sandbox, I want to say, what you are describing sounds like a failed sandbox to me, not a well run one. I mean, if you are running a game and nothing is happening, then that is a bad session (unless you literally have five players totally content to do nothing). This is something plenty of sandbox GMs have written about and talked about. I would point people to Bat in the Attic (Rob Conley's blog) for some good advice on that. I've played in Rob's games and seen his advice in action. It is easy to mischaracterize his advice, or misunderstand and assume it leads to what you are talking about, but if you pay attention to his real points, you see he advocates avoiding that very problem. In fact, one of the things he advises is throwing more hooks and leads for groups that might not be as accustomed to taking initiative. I played in a real gritty medieval adventure with him, which is a campaign concept that could easily fall into the extreme you laid out. However there wasn't a dull moment. I worked out my character background with him. We tied it to the setting material (which was important because this was an attempt to do an authentic medieval campaign), and it worked great. He didn't shy away from making our backgrounds relevant. It is just rather than have us declare things and those things be reality, he fit our concepts to the setting and brought the setting to life enough that we could use our backgrounds well in interactions. We were free to do what we wanted, but stuff still happened in the setting.
A sandbox isn't supposed to be static. There are pages of advice online regarding this. And there are any number of approaches to handle it (from countdown clocks to adventure seed tables to encounter tables to world in motion). My advice to anyone who is thinking of playing or running a sandbox but might be hesitant is to get information from the horse's mouth. Go to the places where people enjoy that style of play and learn what they do. Getting that kind of information from a thread like this or a venue where it isn't really the norm, is sort of like me getting all my information about narrative play from a sandbox GM or forum. You are going to get a misleading perspective on the matter (not saying you are doing that as I don't know your background with sandbox play, just making a general point).
That said, if you don't like sandbox, it might not be for you. If you don't like GM as final arbiter, it may not be for you. These are just play style preferences. And they don't have to be all or nothing. You can easily take elements of a sandbox and mix them with other things if you like the idea of the openness and freedom but worry about the adventure not having enough sense of direction or excitement. Also, you can honestly run a sandbox with any kind of 'setting physics' you want. A lot of people run settings like they are the real world. Not everyone does. I run my wuxia campaigns using wuxia logic (my players like to say "Chang Cheh physics are now in play"). This means it isn't at all unlikely that when they sneak into the brothel ship to investigate the sleeping weasel of a scholar who has been spying on them, a bunch of sect henchmen jump out from the cabinets and attack. A lot of my choices in this kind of campaign are often guided by that kind of genre logic. I am somewhat sparring in its use. I try to give it the feel of a grounded wuxia. But it still has dramatic flare.