Not only do you keep the room more dimly lit, also keep the in-game settings more dimly lit. Use fog, rain, mist, glare, snow, darkness, etc. to limit field of vision.
Describe the age, weariness, and disrepair of buildings, streets, etc. Highlight soot, puddles, dripping sounds, rust, knocking pipes, etc. In model railroading (my other hobby), purists weather everything with paint or chalks, do the same with your descriptions.
Use unsympathetic authority figures, ambiguously corrupt politicians, apathetic citizens, etc.
One effective way to scare them is to mess with the ordinary, everyday things. Their food tastes slightly metallic, milk is a bit sour, bread is stale, beer and soda are flat or don't make a head, things settle out of solution too quickly, the car doesn't start, or stalls at an inopportune time, in a park at night they see the headlight on a kid's bike dim from top to bottom like it just winked at them. Use little things to mean a lot.
An example of how SLIGHTLY changing the expected, ordinary things throws people: Early in the twentieth century, there were travelling hucksters who would come into town and challenge the best baseball team in town to a game, sell the entire town on it and have their significant other collect admission. The night before the game, he would go out to the field and move the pitching rubber slightly forward or backward from its original position, possibly including (if it could be done without notice ) realigning the basepaths to account for it, moving home plate, etc. Then he would prove to be an unhittable pitcher in the game, because the rhythm of the town's baseball heroes would be off slightly because of the distance change. They would rarely make contact with the ball.
This is what you have to do to the players. Throw off their rhythm. Alter descriptions of normal things slightly. They may not even consciously notice, especially at first, but when their expectations of what things will do or how things will behave are thrown out of rhythm, it will scare them but good.
Also, measure your speaking rhythm. Speak softly and slightly slower than normal. Use lots of little differences in inflection, tempo, timbre and pacing. Several posters have said that the players have to want to be scared, but in addition, the GM has to feel secure enough with himself to do things that may make him or her feel self-conscious.
BTW, I am going to be running a fantasy horror game this winter (after Heroes of Horror is released). This is a great thread.
DM