Hussar
Legend
Arrgghh. The dreaded Paladin trap. The death of games since 1976. 
As someone who actually likes playing paladins, I've seen this sort of thing more times than I can count. Honestly, from what you've stated here, it's not like his actions cannot be justified. The mayor is ordering a DEVIL around to maintain law in the town. The paladin can pretty easily justify attacking the mayor here, even if the mayor is just a victim. Paladin's can kill non-evil and even good targets in such a case.
As far as exhaulted feats go, meh, again, don't punish him for making choices. Let it ride. Or, if it really bothers you, take the book out of the mix. It won't end the game to do so.
My big question is, how did you have the NPC's react to all this. The paladin just attacked the leader of their community. That should have consequences. Not, "Ok, go to jail and have a time out in the corner" kind of consequences, but actual role play stuff - people won't talk to him, they cross the street to avoid the "enforcer style" paladin.
In other words, if he wants to be a Hanging Judge sort of paladin, treat him as such.

As someone who actually likes playing paladins, I've seen this sort of thing more times than I can count. Honestly, from what you've stated here, it's not like his actions cannot be justified. The mayor is ordering a DEVIL around to maintain law in the town. The paladin can pretty easily justify attacking the mayor here, even if the mayor is just a victim. Paladin's can kill non-evil and even good targets in such a case.
As far as exhaulted feats go, meh, again, don't punish him for making choices. Let it ride. Or, if it really bothers you, take the book out of the mix. It won't end the game to do so.
My big question is, how did you have the NPC's react to all this. The paladin just attacked the leader of their community. That should have consequences. Not, "Ok, go to jail and have a time out in the corner" kind of consequences, but actual role play stuff - people won't talk to him, they cross the street to avoid the "enforcer style" paladin.
In other words, if he wants to be a Hanging Judge sort of paladin, treat him as such.