The roleplaying went that way, but then a player looks at his sheet and says, "Hey, I've got a 16 CHA, and I'm +11 at Diplomacy...I shoulda convinced that guard to let us in!"
But, he didnt'. When we roleplayed it out, although it was an enjoyable RP momemt. The players even laughed at my rendition of the guard. The juxtapositon of the characters stats and the outcome of the roleplay may not seem correct.
In my view, the character did what the character did as the player played him....Thus, this is the problem I sometimes have with the game.
I think that the RAW charisma/diplomacy rules are missing some details that should clarify this situation.
First of all, Diplomacy per the RAW can't be used to convince someone to do something for you. It's frequently used that way (and rightly so I think), but that's not what it does according to the rules. So understand to begin with that if you are using Diplomacy in that way, you are already off in house rule land. Diplomacy per the rules only changes the attitude that the NPC has toward you - perhaps changing the gaurds attitude from indifferent to helpful.
However, and this is important, it's not at all certain that a person who is helpful will automatically do X. In this case, the gaurd might helpfully let you in, but he might also consider that he's risking his job by letting the players in, and so indead helpfully directs them to his boss or relays the request to his boss. That is also 'helpful' (at least from the NPC's perspective, the PC's of course may not want him to 'help' in that way at all, in which case, perhaps we should be examining how 'bluff' works).
Secondly, if you are using Diplomacy to do persuasion checks, keep in mind that there can be massive circumstance modifiers applying to either the check to modify someone's attitude or the check to persuade them. In particular, trying to persuade someone to do something that seems to be against their best interest should have a -10 penalty or higher. If the guard evaluates that letting the PC's in is against his best interest (he could get beheaded for violating orders), then apply a -10 circumstance penalty. Likewise, if the NPC is under orders from someone of higher rank than the PC's, this could also apply a circumstance penalty. If the King specifically ordered the gaurd to not let anyone in, and the PC's are strangers, that might be worth another -8 circumstance penalty. If the NPC's deity directly ordered him not to let anyone in, and the PC's accent identifies them as coming from a nation currently at war with the NPC's nation, that might be worth like a -15 circumstance penalty ON TOP of the -10 penalty being applied because the NPC is risking his life by cooperating.
In otherwords, depending on the situation, it may be perfectly reasonable that a mere 16 CHR and 11 ranks in diplomacy aren't enough to get someone to open a door, and the outcome of the roleplay may be perfectly reasonable for the skill level indicated. On the other hand, the same outcome might be rather unreasonable if the PC has a +30 bonus to diplomacy and the player makes a reasonable argument that appeals to the NPC's values.
Or consider a slightly different situation. Perhaps the guard was ordered by a sergent, a man he doesn't particularly like, to not let anyone through the door. And the PC's are clearly low ranking aristocrats of some sort, and they want through the door. You as the DM may now evaluate the circumstance entirely differently. The NPC may now consider that the PC's outrank his sergant, and that the trouble the PC's may cause him if he doesn't open the door is more than the sergant can, and further that letting the PC's in my discomfort the sergant (to the NPC's own amusement). In this case, you might consider a circumstance bonus of +4 or so on a diplomacy check, with the result that very diplomatic PC's could hardly fail provided the player offered in a mildly good argument.
But in every circumstance, play out the encounter. You know more earn a diplomacy check by stating that you want one, than you earn a climb check without specifying which wall you want to climb and how you get to over to it to climb it. Diplomacy checks have to come with some sort of explanation of the content of the request or encounter. For example, the outcome of a diplomacy check where the player says he is flattering the pretty merchant's wares might be very different than one where the player says he flirts with the pretty merchant. And of course, it is not enough to say merely, "I want to flirt with the merchant." This must be met with the request, "Ok, what do you say?", because the results of saying, "My good lady, are you not looking radiant this morning?" or "Lo wench, let us retire to yonder ally, for you make me randy." can be very different. Depending on what you know about the pretty merchant, and how overt and aggressive the flirtation, flirtation might add a large circumstance bonus or a huge penalty to a diplomacy check. Content always matters, regardless of whether you throw dice.