Another related idea, that comes through in
@Lanefan's posts, to an extent in
@CreamCloud0's, and maybe also in yours and
@Xamnam's, is this:
If the GM imagines to themself, without sharing it with the player, that the reason for such-and-such event that has been described at the table is such-and-such other event that no one is imagining except the GM, then
that is part of the "scope" of the setting.
For instance: if the GM describes to the players that they see some beggars by the city gates,
and the GM thinks to themself but doesn't say to the players, "The reason for those beggars is that they've been driven off their farmland by the evil overlord", then part of the setting includes
that the evil overlord has driven people of their farmland, turning them into beggars.
I think of the GM's private imaginings as possible tools, and prep, for saying things that become part of the shared fiction. But it seems to me tautological that, being private, they are not shared, and hence are not elements of play per se.