You're probably already aware of them, but BEOWULF: Age of Heroes and The Lord of the Rings™ Roleplaying are two systems to look at, at least to see my thoughts on how to do it.
Short answer, feats and widgets to allow very contained magical effects on the player side. Monsters need...