You never stop getting warlock abilities once you have them.
You never stop getting paladin abilities, well OK you can become an Oath Breaker and get a few different subclass abilities.
You never stop getting cleric abilities.
The patrons, gods, forces, what have you are all fluff and don't matter to the mechanics of the character. If the fluff changes, change it. Your character pisses off one deity only to be recruited by another stuff like that, or the power you tap into changes to not coming from a being with whims of fancy but to cosmic power accessible by all.
Paladins, clerics, and warlocks don't have alignment restrictions for a reason. The atonement spell no longer exists for a reason, because it is no longer necessary.