I think this is time for a story-based penalty rather than a mechanical one. Also, I personally think revoking their powers is a tad harsh on the player, as it could remove the things they found fun about their character.
Possible real life model: in Catholicism, a priest can be “laicized” for egregious behavior, which strips them of their rights and obligations as a priest, making it impermissible for them to do things like hear confessions or say Mass. However, because Catholicism considers priesthood to leave “an indelible mark on the soul,” laicized priests technically still have the ability to do these things.
In light of the fact that Herald oaths are tied to their archetype, what you could do is introduce a punishment if Heralds use an ability tied to their archetype. Green Knights must dismiss their familiar, Inquisitors can’t use communication spells, etc, without some kind of story cost. However, they still can do such things, and may be tempted to do so. However, if they want to redeem themselves, they still have all of their vanilla Herald abilities to make that quest possible. This also has the advantage of not needing to edit their character sheet.