For me, its the subtle things that can't be explained. For example:
* a shadow moving when nothing is there to cast the shadow
* leaving a room in a building to come back in and find something substantially different (like curtains taken down and folded), even though you were in there less than a minute ago and no one else is around (yes, that really happened)
* the flame on a candle extinguishing and then re-lighting by itself a few seconds later
* the sense of being watched
* finding things that seem to be coincidences, but have too much similar between them
* isolation- its always used in every good horror movie/book/adventure. If the PCs can go somewhere for help, it ceases to be scary
* deja vu- this once can be very effective if used on PCs several adventures in advance. Give the PC a brief image during an adventure of something totally out of context, then a few adventures later, present that situation to them, only it is relevant now. It really creeps people out.
* losing control of your actions/sanity- somebody else said this too- but I can't stress how terrifying this is to most people. Make one PC see and hear things that are not there, while the others have no idea what is going on. Guaranteed creep-out