What about having Wizards, but they don't necessarily have magical ability.
Wizardry could be the studying the theory of magic. They do complex calculations and develop new theoretical spells. However, to cast magic one has to have an innate ability to use magic.
So you could have someone be...