I think this will work great if the PC's have a reason to be in the place other than killing the monster. And you need ways to split up the party, such as a sequence of events that the party must cause within a tight time frame, or traps that isolate party members, or even better - a combination of the two. The Aliens movies were great about this.
Picture a group of adventurers exploring a long-forgotten ruin with cryptic heiroglyphs and bizarre construction. The ruin is oddly deserted, and they easily (almost too easily) make it to the heart of the ruins. There they discover that there is no treasure (perhaps the treasure map they had was a decoy), but through environmental clues such as dismembered skeletons, ancient bloodstains, journal fragments, and deciphered heiroglyphs they piece together that the entire place is a trap to feed hapless souls to Nameless Evil (NE for short).
Quickly, the party tries to find its way out but, oh crap, they set off a trap on their way in and the exit is blocked. What's worse, they find that their eyes are stinging and lungs are beginning to burn. The air is becoming slowly more caustic. In just a few short hours, they will all be burned to death by acidic vapors! As the party is trying to find its way out, they stumble on a previous victim with a partial map that shows what might be a control room that can stop the gas. It also shows an alternative exit through underwater passages, but in order to pass through there they will need some sort of breathing aid. Fortunately, further examination reveals an area that might contain scrolls of water breathing, but alas, it's on the opposite side of the ruins. There's time to get the scrolls, get to the exit, or get to the control room, but only one of these goals can be reached before the vapors burn them alive. Immediately thereafter, a party member falls through a locking pit trap into a labyrinthine sub-level. Just then, NE makes a cameo as nothing more than shadowy movement and hissing noises fading into the darkness.
Uh oh. Does the party go rescue their teammate, do they chase after NE, do they run for the exit, run for the scrolls, or run to the control room? There are all sorts of other complications you can add. Perhaps a traitor steals a MacGuffin and runs to the exit, threatening to close it behind him forever. Maybe the NE needs just two more victims as sacrifices before it can unleash a horde of NEs on the world. Whatever it is, even though the NE is the primary source of the terror, the desparate situation is what makes the party split up.