I'm still trying to achieve that happy balance between improv and preparation. I would love to prepare tons of material, but I never find the time, between work and evil things like Final Fantasy X. And improv works well in my games.
I think the only reason improv works well for my established game is familiarity. I _know_ the characters implicitly. I know how the players play them and the likely routes they are to take. Plus, the base city is well-defined in my mind (and hopefully in the players as well). So when they go looking for odds and ends, I can usually rattle off interesting details and roll with that.
The way is do most of my pre-game planning is by driving. I have a long commute to work (nearly an hour each way), and while listening to my music, I visualize certain scenes that I want to see happen. I plan out parts of important conversations. I bought a digital voice recorder so I could take notes (pen & paper became hazardous on the road). During the games, I play games of 'connect the dots', subtly guiding players to scenes that I've already imagined out.
It works out well, but I've started a second campaign and have found that I need to work a lot more until I get as familiar with the group.