What's in question is how many of those reference points you can remove and still have a functional and enjoyable game.
Say something to prompt someone else to say something might be a perfectly functional, enjoyable game. But it won't be the same game as any of the RPGs I enjoy.
First I don't experience the type of games I sometimes play--OSR dnd type games--as being a "story told by someone else" or that I tell to someone else. Someone else might look at the gameplay loop of a dungeon crawl (for example) and find it lacking in player agency because the player can only announce their actions. To me, that's an over simplification to say the least; I suppose I can see how one formally arrives at that position, but in practice is not how I experience those sorts of games.
When you play a dungeon crawl, is the GM allowed to change the dungeon map at will? If you have entered a room via an open archway, and then declare that you leave the room the same way, is the GM free to tell you that you don't leave? That you suffer a leg cramp and fall to your knees?
The dungeon crawl games I'm familiar with (Moldvay Basic and AD&D, which many OSR games are based one) do not follow you posted play loop. They go something closer to this:
1. The DM describes the environment.
2. The players describe what they want their characters to do.
3. The DM refers to the map and key.
4. The DM extrapolates from the map and key where the PC goes (if moving) and/or what bits of architecture, furniture or similar that the PC discovers and/or touches. If the DM is not clear about what the PC is doing relative to the geography and architecture, the DM might seek clarificatoin from the player.
5. The DM calls for an appropriate roll if a relevant subsystem is triggered (eg opening a door; climbing a wall), or makes an appropriate roll if a relevant subsystem is triggered (eg searching for a trap or secret door, or listening at a door), or extrapolates the immediate result as faithfully and neutrally as they can if the action is moving things, lifting things, poking things, etc.
I think that's still probably incomplete, but is closer to what Gygax and Moldvay describe in their rulebooks.
there are games that more or less put the establishment of the fiction in the hands of the dm, and the players play particular characters, usually just one at a time, in that fiction. I think it's an exaggeration to claim those games are "zero agency," because the players have control of their characters, and confusing because within that framework the term agency has a different and more specific connotation than "establishing the fiction" of the world. (btw if "low trust" is pejorative then so is "zero agency" imo)
I don't want to be mean, but this is just wrong for huge swathes of RPGs, including so-called "trad" ones.
In Rolemaster, establishing the fiction is not all in the hands of the GM. Eg if a player has their PC talk to a NPC, then if the upshot is contested (eg the player wants to befriend the NPC and the GM doesn't just go along with this) there is an expectation that the Influence and Interaction table will be used. This, in turn, dictates how the NPC responds. Ie the GM has neither sole nor unconstrained authority over the fiction.
Many similar examples could be given for RM, and for other games of a broadly similar character.