A good rule of thumb which I use to adjudicate Diplomacy is to ask yourself what requests you would agree to from a random stranger off the street (Indifferent), an ordinary friend (Friendly), and a close companion (Helpful). Diplomacy is not dominate person. It doesn't make everyone the character meets into mindless automatons.
If you asked a guard to let you see what he's guarding, an Indifferent guard might tell you to push off, a Friendly guard might turn you down nicely, and a Helpful guard might bring the matter to his superior. Using diplomacy to get the BBEG to give up his grand plan is probably useless, especially if he places no value on friendship.
When there is a character who is good at diplomacy, the DM should plan ahead and decide how creatures and NPCs will react, and what help (if any) they will give at various reaction levels. The character should get some value from his investment in the skill, but it doesn't always have to go his way.