I'm not sure that your confusion is entirely warranted.this confuses me from a RP perspective. If my PC genuinely considers they are being threatened by an overwhelming force intent on something nasty and potentially apocalyptic, they aren't going to want to play nice.
At least as they operate at my table, these "gentlemen's agreements" are between the players, not the PCs - they operate at the metagame level.
A simple example from my 4e game: the player of the polearm fighter and I have an understanding that of two somewhat broken feats - one that lets him immobilise any marked target whom he hits with a basic attack using a two-handed weapon, and one other one that gives him forced move on OAs (which with his Rushing Cleats and Polearm Momentum would mean his OAs knock foes prone) - he will only take one.
At the PC build level, this sort of thing is fairly straightforward, I think. It's sort of an abbreviated or ad hoc form of houseruling by consensus.
But it can even work at the action resolution level.
In Rolemaster, for example, there is a type of scrying guard that creates a false image when the target is scried upon. One of the players in my RM game worked out that the PCs could use this to send messages to one another - set up false images of holding a sign with the relevant message, and then use a scrying spell to get the image. I can't remember excatly what the scrying spell to be used was, but it was something lower level (and therefore easier/cheaper to cast) than long range telepathy.
We thought about looking for ways to change the wording of the scrying guard to rule out this use of it, but nothing straightforward suggested itself. So everyone at the table just reached an agreement that they wouldn't do this sort of thing, and that scrying guards would continue to be used simply to guard against scrying.
Where gentlemen's agreements can't work, in my view, is when they would have to go to the very core of a PC's abilities and functions. Because at that point they would require the player to hold back when playing his/her PC, not just to avoid some marginal exploit, but just at that point where the PC should be firing on all cylinders.