You certainly do not have creative agency in the real world. If you thought that was a rebuttal, you are grossly mistaken.
Agency as a player can involve things that would be completely impossible for a person to do in their own, natural life. Just as every other part of a roleplaying game can involve things that are completely impossible for a person to do in their own, natural life.
Did you seriously think that "just as in real life I have no agency because I can't imagine that museum and have it pop into existence" would be a rebuttal when we literally are talking about owlbears and elves and invisibility potions?
I have agency in real life. Players in my games have agency even though they cannot add to the fiction of the world outside of their players. The level of agency is different, but it's not directly comparable.