I think that might be a little TOO restrictive.
I don't mind a little dabbling, but I don't like that the specialty doesn't have any mechanical weight beyond some cheaper copying. How about, instead of a unique spell list, the specialists just have a unique progression? Instead of automatically adding 2 Wizard spells to your spell book every time you gain a level, you only get 1 wild and the other has to be from your specialty, with a few levels where you can only pick 2 from your specialty? This would naturally skew your spell list towards your specialty, but still allow some flexibility to make interesting builds.