It depends a lot

. Players love rolling dice, but sometimes it's just best they don't know the result. At the same time, often the characters could be able to percieve that they have performed well or badly, like a character rolling 1 in Disguise may notice that his face looks totally fake. A couple of examples...
Move Silently - Normally it's ok to let them roll openly. A character is trying to sneak past the guards, player rolls a 1, the character noticed that he was not very silent, perhaps he dropped his shield or hit an object on the floor... it doesn't really mean that he failed, because perhaps the guards rolled even worse, but he knows that he wasn't as silent as he wanted. Often the consequence of a failed Move Silently check is immediate and there's no point in hiding the result.
Listen - (1) The PCs roll to hear the ambushing bandits are about to attack (you want to determine if they are surprised or not): open roll since in one way or another, they are going to find out!
(2) The PCs roll to hear someone invisible following them: hidden roll since it may not have immediate consequence (the invisible creature is not attacking), and it's important they don't have any clue. In this case you are not even letting them know that you are rolling a Listen check.
(3) The PCs are eavesdropping at a door to check is someone is in the next room: hidden roll, although it probably doesn't apply if they are taking 10 or 20.
Search - IMHO it's best to keep it always hidden. This skill is most often used with take 10 or 20, but when it's not it's too easy to roll again if you see the result was low.
Other skills, and accounting
what happens if you know you roll badly (it's important that if you cannot retry, knowing you rolled a 1 doesn't help you; if you can retry, it will definitely make you reroll, but also you could always take 20):
Appraise - Open:
you know you are not very sure of the value (the % result is still hidden instead)
Balance - Open: no need to hide
Bluff - Open: the opposite check can be hidden in case you want to hide the effective result;
you know you could have spit a better lie
Climb - Open: no need to hide
Concentration - Open: no need to hide
Craft - Open: no need to hide
Decipher Script - Open: the PHB says to hide, but it's enough to hide the Wisdom check;
you know you don't get the meaning
Diplomacy - Open: the opposite check can be hidden in case you want to hide the effective result;
you know your arguement was not the best
Disable Device - Hidden/Open: I would make it case-dependent and give clues;
you may know the trap is still working
Disguise - Open: the opposite check can be hidden in case you want to hide the effective result;
you know your disguise is approximate
Escape Artist - Open: no need to hide
Forgery - Open: the opposite check can be hidden in case you want to hide the effective result;
you know your document looks fake
Gather Information - Open: even if you fail it takes a long time to retry
Handle Animal - Open: no need to hide
Heal - Hidden/Open: I think this one is more subjective, depends if you want to let them know the disease it's over or keep the suspence
Hide - Hidden: it's more difficult to self-evaluate than Move Silently; if consequences are immediate, roll it openly;
you may know your camouflage is approximate
Intimidate - Open: no need to hide
Jump - Open: no need to hide
Knowledge - Open: no need to hide,
you know that you don't know; some use a house rule of giving false information in which case you may want it hidden (but I don't recommend this HR)
Listen - Open for surprise, otherwise Hidden: see above,
you know you made some noise
Move Silently - Open: see above
Open Lock - Open: no need to hide
Perform - Open:
you know how you sang a note out of tune
Profession - Hidden/Open: very dependable... sometimes a professional notices his errors when it's too late, but normally in D&D you use this skill for immediate clues on something rather than to make a living
Ride - Open: no need to hide
Search - Hidden, but easily circumventable by Take 10/20
Sense Motive - Hidden/Open: normally I roll SM myself as a DM, but in some cases it could be left open;
you may know you don't have a clue of what's going on
Sleight of Hand - Open: the opposite check can be hidden in case you want to hide the effective result
Spellcraft - normally Open except a few specific cases
Spot - Open for surprise, otherwise Hidden: see Listen
Survival - Open: you could make it hidden for weather forcast, if you want it funny

; actually you may want to keep it hidden for tracking (as Search)
Swim - Open: no need to hide
Tumble - Open: no need to hide
Use Magic Device - Open: no need to hide, unless specific devices have long-term side effects
Use Rope - Open: the opposite check can be hidden in case you want to hide the effective result;
you know the rope is loose