Speaking of video games, a lot of players (probably including some in this thread) would likely be shocked at the amount of manipulation and deception many games use. For example, I know of plenty examples were developers or studio/publisher reps lie through their teeth about how their game AI works. Baldur's Gate 3 and their Karmic dice are a very mild example of this kind of manipulation (especially as it's mostly out in the open).
Anyone who thinks they could not enjoy a game or feel agency playing it if it was based on illusion should be very glad they can't see what's inside the video games they play. A lot of people tend to angrily refuse to accept this information and/or choose to believe their favorite games are among the ones free of deceit and illusion. But rest assured it is very much possible to feel agency that goes beyond your actual capacity for impact on the game state. Part of the trick is to keep it hidden. I honestly don't understand the angry dismissal that perceived agency isn't valuable. I can understand and empathize that some prefer games (video or ttrpg) where they can be sure there is no illusion going on. But so much entertainment is based on illusion.
Perceived agency is a relevant type of agency to consider, as that's the only relevant kind of agency for a lot of players. Whether the player action reveals information or impacts game state according to a truthful hidden state tracked by the GM or whether there is some element of illusion or ad-hoc GM state manipulation is irrelevant to these players. The perceived agency is what matters. For example, I've had players who among themselves reached the conclusion about some aspects of a hidden piece of information - and I've changed the hidden state to make their presumptions real, added some hidden information they're still speculating about - those sessions sometimes then branched off into some amazing interactions with the unknown consequences of their assumptions.
If I had stayed "truthful" in all of these cases, the sessions would have been boring - the players would have been disappointed their conclusions were mistaken. Letting players build the illusion in this manner is useful and valid. In a sense those players helped author the shared fiction, but they didn't know, and I'm pretty sure they didn't want to know. There have been many tables I've played at, on both sides of the screen, where the consensus was that illusion wasn't just ok - but commendable if it improved the experience. It doesn't work for jaded cynics, hardcore gamists/simulationists nor for many of the narrativist styles. But it can work and it is a valid kind of agency. It isn't purely passive and imagined agency - the way players interact with illusions shape the experience and the narrative.
The more involved players get in this kind of style, the more impact they have. Player agency is exercised through the obligations of the GM to bend reality around player actions. Some might object to the narrative power this places with the GM - and others might object to the unreasonable expectations and burdens this can place on a GM. If "truth" is never an excuse for a session to be somewhat boring and not have enough of whatever the players want (be that combat, roleplay, puzzles, investigations, etc) it puts a lot of pressure on the GM to be able to improvise quickly and to do flexible prep. But some players accept that the GM has this power and some GMs are find with wielding it. The appetite for how much illusion is acceptable in relation to the amount prepared game state also varies - so it's not like every table I've played at is either fully behind illusion or fully against it. In particular the acceptance of the degree to which the GM is allowed to thwart or subvert player expectations / expression is a point of differing taste. As a side-note it is my impression that the taste for illusion had declined over the years, but that's not something I can base on any empirical evidence.
Anyways, this is just one example of a playstyle which has player agency but doesn't adhere to the notion of the game revolving around highly codified authorship of shared fiction (with or without distributed authority) nor does adhere to the notion of a truthful hidden game state prepared by the GM. And there are other variants. There's ritualized styles where the agency is to some degree attached to idiosyncratic rituals of play rather than directly to narrative or game state. There's power fantasy games (incredibly common I've found) where player agency is very self-centered and almost solipsistic.
Finally I want to emphasize once more, I am not saying players who have a narrativist view of player agency are silly malcontents that should stop enjoying their play style or trying to win others over to it. Nor am I saying people who find the notion of perceived agency distasteful should stop feeling that way. I am just saying that player agency is as varied as players themselves. Which is also why I think any GM who has problems with players complaining about agency need to focus on the expectations and wishes of those players - and not to try and get into a debate with the player about the definition of agency.