So are we also restricting opposed schools? I think wizards get too many slots now, excluding anything flavourful, but if they took the hit of not having access to some spells I could support adding some extra flavour-rich spells
No, I wouldn't. Why take things away? Give them both. You could also raise the DC for spells of their chosen school, to make them better at those spells than other casters.