I'd say that there may be an Eberron style explanation where disjunctions between the planes cause occassional unusual stresses that empires find difficult to survive.
Alternately, it's actually weird conjunctions of the planes that make it possible to form empires at all. Then it's just a...