Yes, the Conan soundtrack as been known to show up during a battle and at other times, LOTR runs during our sessions, but most of the time I use special sound effects CDs that one of my players creates for the group.
So far he has made a couple of different forest/wilderness tracks, an underground cavern track, and my favorite... the Tavern. The Tavern track has the sounds of people talking, drinking, moving about, and having a good time. It's looped, so every so often you can hear what sounds like a halfling climb up on a table, say a few things, fall off the table, and break his glass. Whether the party is at the Dragon's Breath Inn drinking and gathering information or they are camping in a dark forest along the foothills of the Stoneheart Mountains, these sound effects CDs really help the "suspension of disbelief" process for my players and help transport them into the world I'm trying to create.
It doesn't hurt that he works in the audio production business
I'm not sure if I should admit this, but I've also been know to throw on those cheap discs you can get at Target for about $10, like Celtic River or the Sounds of Ireland. They work for my group. They aren't distracting to the game and they add a nice background sound to the session.