The logic of the situation, for me, has always been an issue of training. Wizards focus on learning how to cast spells. Warriors focus on arms and armor. If you let wizards acquire a major skill of warriors without detracting from magical focus then why can't warriors just pick up magic on the side?