You know I keep seeing the assesment that players aren't into the "background" of a setting and that it is usually wasted upon them. If this is the case why are rpg's based on settiings made and sold. Star Wars, Exalted, Qin:Warring States, etc. are all games that have an inherent world attached to them, and that players enjoy interacting with. In Star Wars a player may not interact with a sith lord or go to the homeworld of the jedi council for many levels, but they're there and this adds to the enjoyment and versimilitude of playing a jedi. In Exalted a PC knows that the Wyld is the purview of the lunars, fae and mutants, they may not interact with them directly in an adventure but if they take a side trek into a Wyld area both the GM and the players know what they're most likely to encounter.
I wonder if the whole "world building is a waste" sentiment could be self perpetuating. Yeah it's a waste if it doesn't capture your players imagination, but if it does all it can do is add to the enjoyment of the game. Maybe the problem is that some people just aren't good at it. I personally am not a fan of dungeoncrawls and feel adventures more so than the actual world should be the purview of a character and their particular desires. What good is it if I've prepared the Lost Tomb of Ansakor, if the PC's instead want to spend a session jockeying for political position amongst the decadent nobles of Algoth Doure?
The world-building gives a canvas upon whic the PC's are free to paint whatever picture they want. A pre-made adveture however seems to have already painted the picture and gives the illusion that the PC's have free will. If all I have for the night is the Caverns of Maegoth Wold detailed then I've essentially forced the PC's to go into those caverns...If I've built the island of Maegoth Wold then the PC's have way more freedom in what type of adventure they want to pursue, they're motivations, passions, and interests direct what part of the island they explore or interact with instead of what I planned for them to interact with.
Is it more work? Certainly. But it's worth it IMHO to give my players that experience of a real environment.