In order to properly scare the players, you have to threaten something they really care about, whether it is the characters themselves, certain NPCs, or a certain plot development.
First you have to create a connection, then you have to get the players to invest emotionally in that connection, and then you have to threaten it.
It's a narrow path to tread; if you press too hard on the threat, you risk breaking that connection and end up with the players stopping caring again. The most basic emotional tie is the one to your character, but press hard enough, and also that tie can break.
I had one campaign that went real sour on me. Conflicts with another player caused my DM to kill of my characters one my one. That's not the important part, what is important here is that after those character deaths I simply did not care what happened to my character at the end. Whatever opposition we met, however big and scary, did not faze me the slightest, nor did taking crazier and crazier risks.
Similarly, at the moment I am playing in a campaign that's supposedly a horror one, I think. We are at least meeting a lot of grotesque gross-out monsters with purportedly scary abilities. But, since every named thing we meet is something you are supposed to kill, without any NPCs to care about, no important plot development to protect, well, it gets curiously like just grinding for money in an MMO...