I pretty much run totally on auto-pilot. I may get an idea for a villain or a setting or a scene that I write down and try to steer towards, and sometimes I'll prep for things that I want to be able to access (ferinstance, I'll prepare certain archetypal NPC types so that I can conjure one up at a moment's notice if need be) if they don't already exist, but everything is meant to facilitate me being able to pull good stuff out of my butt at a moment's notice.
So what makes for a good "improvisational" GM? In my experience I've found that it really depends on...well, experience. If you're really into the settings and rules, and you're really warmed up from regular GM-ing, then it's easier to improvise than if you only run a game every once in a while. I think of it this way--GM-ing is a skill which must be practiced regularly, and it comes more naturally to those who do so. Those who don't--or, in my case, can't--practice regularly benefit from lots of preparation.
There's a few tricks to running games that are heavier in improv that anyone with the inclination can adopt.
Archetypes are your Friend. We've all consumed enough movies and books and TV shows to know how some character types behave. It doesn't take much of a stretch to put that into a D&D game. If the sullen warrior of amazing skill feels insulted, he's taking his magic doodads and going home. If the kindly old priest sees someone in trouble, he'll try to help them, no matter how outmatched he may be. If the dangerous woman notices someone she can manipulate, she'll lure them in with promises of everything and only deliver them death. It pays to work against archetype maybe 1/4 times, to keep 'em guessing, but you can go a long way with these parallels.
Let the Players Drive It. You don't need an answer to a question until it is asked, and once it is asked, you have to make a decision between it being common character knowledge (and then just telling the players the answer that makes sense to you at the time) or not (forcing the characters to research or go ask the right person). At the same time, your characters need to have goals and dreams and hopes that they will pursue.
Anything can spin off into a quest, not everything should. The basic rule is, "If the PC's want something out of the ordinary, they need to work for it." Think of the types of quests in a game like World of Warcraft: you can go slay x number of creatures, y specific creature, collect z number of items, a specific item, etc. All of these basically share the type of "NPC needs something done in dangerous territory." If your players seek something important, tell them they have to go prove themselves, or that the NPC needs special unguents, or that research cannot proceed until some ancient family enemy is slain. In this way, you can "nest" quests, too: To give them the information, the NPC is going to need a ghost haunting his family's old mansion slain. To slay the ghost, the PC's need to research the family history at the library. But they're not citizens, so they can't get in. How do they solve this problem? It's not good to ALWAYS do this (because PC's get frustrated if success is denied for too long), but it lets you pull an adventure out of anything.
Ask questions, only give answers if you have to. When the PC's have goals, they are the ones who need to accomplish them. A PC tells you "I would like to be the greatest warrior in the land!" you ask them "How?" they tell you "I'll slay the current greatest warrior in the land!" From that line, you should have half a million archetypes running through your head already: the arrogant swashbuckler, the unbeatable tank, the mysterious swordsman, etc. Any of these characters could be the current Greatest Warrior. The PC wants to fight them. You need to make that happen: the mysterious swordsman travels alone, so the PC needs to do some reasearch to find him, go into dangerous territory, perhaps find a magical sword that will aid the combat, etc., etc.,...all of this just by asking "How?" Be prepared to prod in one direction or another if they ask you "How do I?" Again, archetypes work as your friend: think of those who have become the greatest warriors before in things that you've read or seen, of how they got there...and think of how to twist them.
Abstract in general, specific in specific. Specificity goes a long way toward creating a believable world, but it's not needed very often. A little goes a long way, and can actually give you further ideas. For instance, you describe a villages houses as "Old. Dilapidated. One has a hole in the roof, another lacks one of the four walls. Rather than being in construction, it looks like they have sat around untended for years." Randomly conjuring that specificity at any random village gives you countless hooks ("why is it in such disrepair? Don't the people care about their buildings? Is something destroying them?"), any of which the characters can explore, if it suits them (or if an NPC needs it). But you don't need to describe the quality of every building, and it's perfectly acceptable to have "Typical Commoner Dwelling" as the norm. If it deviates from that, the PC's may consider it important, but if EVERYTHING is important, then you always mention more than they can explore. Which is good -- it gives them the sense that the world around them isn't ABOUT them. And if the characters never learn what did that, that's fine -- something they don't show interest in is something you don't need to think about, either.
There's more. There could be a whole thread about good improv DMing techniques!