True, maybe whichever way you go you'll have to sort out exceptions.
In some cases you might be able to use a caster level based on an item: uncommon, rare, epic, legendary might be care level 3, 9, 13, and 19 or whatever the minimum caster level needed to cast a spell from it. Otherwise...