Well we somehow manage it in real life, where we only ever get to control ourselves, we do so by interacting and investigating and making actions at the things that we want our world to involve more of
How do you drive a car if you are not allowed to touch the steering wheel at all?
The things you are describing are not driving the fiction. They are certainly other things which may or may not be agency. But they are, often,
not being in the metaphorical driver's seat of your life. Many people IRL have their agency very heavily curtailed, so the fact that IRL people
sometimes have agency is not really a rebuttal.
Edit: The car (or carriage) metaphor is more apt the more I think on it. A "backseat driver" is a known concept, someone who (attempts to)
dictate where the actual driver goes. The driver, unless literally coerced or under some social requirement (e.g. a chauffeur), has no obligation to do any of that. If they
happen to drive to where the "backseat driver" wants to go, it is not because the person in the back seat actually did any driving. It is because the driver decided something and that something coincided with the back seat person's interests.
Conversely, consider something like "driving," or rather
piloting, the starship
Enterprise. Sulu is the helmsman, but he directly relies on other command officers to do his job. He relies on Scotty to maintain and manage the engines (hence Kirk occasionally commanding Scotty to squeeze out more power than the engines are intended to produce), without which no motion is possible. He relies on Spock to report and analyze the sensor readings, without which he would be flying blind and ignorant. He relies on Chekhov for threat analysis and tactical information, without which the ship could not evade danger. He relies on Uhura for communications and computer support, without which his instructions to the computer would be useless.
Sulu has the direct hand on the metaphorical tiller, but four other officers are also touching that tiller in some way, at some time. That's literally part of why they are present on the ship at all. Each performs a task that is materially part of piloting the ship.
So, when the jobs of Scotty, Spock, Uhura,
and Chekhov are all performed by Sulu, and some non-officer person says, "Sulu, you should fly to Vulcan," who has agency there, if agency means driving (or piloting)? Sulu is under no obligation to obey suggestions from lowe-ranking crew. He is already assuming the duties of every officer who has any material effect on the outcome. What "agency" does this random crew member have? The ability to walk the ship until it reaches its destination? Even an animal would have that.