I think the mutually assured destruction theory and the there's always someone more powerful than you theory work just fine. In the modern world, we have the technological capacity to destroy all of civilization with a relatively modest amount of effort. This hasn't happened, for essentially those reasons: the big powers know they can't use weapons beyond a certain level of destructiveness without risking reprisal, and they work very hard to keep those weapons out of the hands of rogue actors who don't care about those consequences. It seems to be working.
And in D&D, you have tangible deities who have real power, as well as epic NPCs, dragons, mystical forces, and so on. While deus ex machina is a term generally used to describe undesirable plot devices, in D&D, it can be seen as a very tangible thing. And again, there are entire books full of deity stats. It's well within the RAW and RAI to assume that level of peacekeeping power exists and is regularly exercised.
It's also well within the RAW and RAI to assume that less omnipotent but still important NPCs understand that magic exists and take countermeasures to prevent it from being used abusively.
Now, do those always work, or can the PCs come up with a plan to achieve their goals? That's where the action is.