In all honesty, probably not. I don't have any issues playing in games where a system lacking casters is used & don't generally mind if a gm disallows certain class/archetype/etc based on interesting lore. A game where the gm bans caster classes is one I would listen to skeptically to decide if the GM understands the crunch well enough to have enough understanding of the problem they think is being solved of if there are interesting lore reasons for it.
Even in darksun where it can be a monumentally bad idea to play a caster doesn't ban casters, it gives cool lore reasons why playing a caster is going to be an interesting ordeal & sets up measures such a caster would need to regularly take in order to protect themselves.