This is exactly what I consider an unwieldy workaround.
I mean, "I rejected the power of the god because the gods are evil and uncaring for the struggles of mortals, but oh, I will continue using spells anyway" even sounds weird. "...but hey, I found a new god to pray to! Overnight!" isn't noticeably better either. Even if the class itself was changed into fighter, well, that opens another can of worms: where the hell did cleric learn all the fighter's tricks? Why didn't she use any of them before?!
Well, the way 4e explained it was pretty elegant, IMO. Gods do not "grant" their power, like some kind of continuous, revocable loan. They bestow a mote of their power...
permanently. Investiture is a completed transaction. That mote of power can then be nurtured, intensified, grown--but it cannot be recalled. Not anymore, after the gods were (more or less) banned from entering the mortal world. A new ritual can be conducted to take the power back, but you have to actually have the person there, and the deity's agents must perform the ritual themselves (it wouldn't be useful if just any old person could do it.) This is why churches have to be really, really careful about who they grant investiture to! Your "faithful" can betray you and yet still wield the power they were given.
That, incidentally, is the lore reason why Avengers existed. They were the "internal police" of the faithful, those trusted with the duty to hunt down (and, if necessary, kill) those who received Investiture and then later betrayed the faith.
Edit: And, to tie this back to what you were finding trouble with, the 4e explanation would be, "Let's give this divine jerk-donkey a taste of their own medicine." This is quite common with "evil" powers, e.g. a Warlock using their patron's dark powers against them, it should work just as well with divine powers. Or you could take Kreia's attitude from KotOR2, where she
hates the Force (seeing it as a willfully cruel, capricious but absentee deity, manipulating lives solely to see more violent wars of "balance" between Light and Dark), but chooses to keep using it in the hope she can learn to kill it, something she freely recognizes may be viewed as hypocritical sophistry.