In an effort to be helpful, I would add these thoughts which intersect with your thoughtful post.
Skilled Play (traditional Gygaxian skilled play, as exemplified by so-called "dungeon crawls" of the 70s) is orthogonal to both System Mastery and to conceptions of Story Now, IMO and IME.
The two primary requisites (or prerequisites if you are adopting old school verbiage!) when looking at Skilled Play are (1) Division of Authority and (2) Preparation. Allow me to explain-
I have ran a few FKR/rules lite games recently in different genres, and a few attempts at capturing a 70s-vibe, Skilled Play "feel" using genre-specific norms. Mostly successful, and one unsuccessful. Doing so allowed me to see what, IMO, allowed for a successful Skilled Play game.
Simply put, it required preparation, and for the narrative control of the world to be within the DM's authority. In saying this, I am not saying that either of these are good things or bad things, simply that they appear to be requirements for this style of play. This is most obvious when looking at the "exploration" pillar of traditional Skilled Play (hereafter, "SP" because I'm tired of typing it) as I will detail further below.
In SP, the conception is that the challenge is not to the character, but to the player (it's a measure of player skill). That's why you see references to the DM as a "neutral referee" or a "neutral arbiter" (whether that can be completely true, is just an aspiration, or was just verbiage is an exercise for people to determine for themselves).
The DM, then, cannot ad hoc the area that is being explored, the DM cannot ad lib, and the DM should not be a fan of the players in SP. The DM is, for all practical purposes, the world that the players are interacting with through their characters. For this reason, the game cannot have mechanics for the players to seize narrative control of the world. The world exists independently of the player's conceptions and desires, and they are using their skilled to overcome the obstacles within it.
For that same reason, the DM must commit to preparation. This division of authority requires trust from the players to the DM that the DM is not changing the world or engaging in illusionism to help or hinder the players. If the players send their characters into a dungeon, there is a map of that dungeon already made. Going west or east will be meaningful choices- they both don't lead to the same ogre. For that matter, the chest is either trapped or not before the players decide to approach it; the pit is 10' deep with spikes or 30' deep with green slime- it's not a Heisenberg uncertainty pit, only to be revealed when an unfortunate soul falls into it.
I understand that when I type this, people will necessarily want to argue with some of what I have said- "What about a spell? Doesn't that, in some way, allow the player to take control of the narrative?" Or, "Hey, if the players are in a town, and a player decides to narrate that there is an inn to stop out, does that destroy SP?" And so on. Obviously, things are always on a continuum, and you can't account for every single use-case.
But IME SP absolutely requires the traditional TTRPG loop of-
1. DM Narrates Environment.
2. Player Declares Action.
3. DM Narrates Effect.
Finally, despite Gygax's 1987 Book (Role-Playing Mastery), neither System Mastery nor System Complexity are required for SP. SP is perfectly possible with either a reasonably complex system, or with a "black box" system similar to what Dave Arneson was originally running.
I know that this doesn't fully engage with some of your issues, but I thought it might be helpful. Good luck with the topic!