That IS assuming such pacts are antagonistic (something that celestial, genie, and archfey warlocks don't necessarily have to worry about) but I don't accept that a warlock's powers are given by their patron, but rather taught or gained but unrevokable by either party. If a patron could just turn off a warlock's power, then the warlock is powerless against them. There are dozens of examples of people who sell their soul and then turn on their patron (Ghost Rider being a good example). I find it far more interesting that warlocks can peace out on their patron and the patron can be antagonistic, but the deal was already made, and the patron can't takeback their gift.
I also am a fan of clerics not losing their powers when they betray their faith because it allows there to be hypocrites, false prophets, and other corruption in priesthoods. If Pelor knows when a cleric is thinking impure thoughts and strips him of his power for it, then you can never tell stories where you need to weed out bad actors in a church or deal with schisms in belief because the easy way to figure out who is acting on Pelor's behalf is to ask them to cast Cure Wounds.
And on a broader scale, I don't like the everwatching eye of Deities and Otherworldly beings always threatening to take character's powers. Mostly because it enables Jerk GM's and has for most of D&D's lifespan. I am fully content with never having Big Brother crippling my character because the DM and I don't agree completely on the tenants of a made-up faith, pact, or oath.