I think rules are there to facilitate the story, and when they feel like they are getting in the way of the logical, natural flow of the action, then they've gotta give. If the players or DM come up with a a creative idea that makes sense in the narrative yet isn't perfectly accommodated by the rules, then that's a problem with the rules. Example: I needed a situation where the players were all unconscious in order to set-up a major story beat at the start of a campaign. I didn't sweat how to make that happen with the RAW, I just made it happen.
On your particular example: D&D rulebooks have always been oddly coy about the authority of the GM to frame scenes, in various circumstances. This creates (in my view needless) arguments, like whether or not the start of A4 or Out of the Abyss (ie all the PCs prisoners etc) is permissible and/or fair. It's not hard for a RPG to actually spell out the rules for this (the three I'm thinking of straight away are Burning Wheel, Agon and The Green Knight).
A more general example:
In Torchbearer, the GM has the authority to declare a player's declared action a "good idea", meaning that it works without a roll. This is a benefit to the player: they get a success without having to make a roll, thus avoiding (i) the risk of failure and (ii) the standard cost of performing an action (in time, or resources, depending on whether the game is in Adventure, Camp or Town Phase). It is also a detriment to the player: the only way to advance your skills and abilities is by making rolls (a little bit like RuneQuest), and so a "good idea" means the player loses that opportunity to mark a test for advacement.
The upshot of this is that the GM, in deciding whether or not something is a good idea, can have regard to pacing, and what makes sense in the fiction, without having to worry about whether they are softballing things. The game has its own built-in mechanism for making the players not want everything to be a "good idea", and hence not feeling like the GM is punishing them in calling for a check.
Burning Wheel is similar to Torchbearer in this respect, except the principles for deciding whether or not to call for a check are not based on what makes sense in the fiction as a good idea, but rather what is at stake in the fiction relative to a PCs' Beliefs, traits, relationships etc.
Apocalypse World is different from TB and BW: there is a discrete list of actions which trigger a roll ("if you do it, you do it") and any other action simply triggers a narration from the GM. There are rules that say how that narration is to be chosen (eg mostly it must be "soft" - ie stepping up the complications - rather than "hard" - ie narrating some irrevocable consequence); there are also clear rules about who gets to say what as the outcome of the dice being rolled.
Thus AW, like BW and TB, doesn't create a problem of "softballing" or "playing the GM" when it comes to adjudication of declared actions. And it's always clear who is entitled to say what, and what the principles are that govern that.
The idea we see in discussions of D&D, about "the rules having to give way to the story", as well as the issues we see that idea give rise to (eg in this thread various posters talking about the players trying to "get away" with things, or get "auto-wins"), arise out of particular design features of D&D, probably the core one being the lack of any general principles about how
establishing consequences relates to
whether or not a roll was made.