When presented with too many choices, my players start bumbling around..
"do you want to join the relief army to fight the orc invasion, use the army as cover to dig into the ashes of your past, follow the trail of the missing abbot, or try and find out why the demand for slaves has increased? I need to know so I can prep the next session"
Evil Halfling, I totally sympathize and understand your situation. I've been there. What I have proposed, my theory, that GMing works better when you focus on developing strong NPCs, is based on a lot of reflection, feedback, and playing in other GMs groups. It took me years to realize.It takes practice. It's a fundamental shift in the way the game is managed. But, I found it ultimately produces more engaging and engaged players, and that means better experiences for the GM as well.
GMs with a strong interest in, and background in, the storytelling arts have trouble peeling themselves away from the script. For GMs like myself who cut their teeth on AD&D 2nd Edition or any of the gaming products of the 1990s, it's particularly difficult. There was a paradigm shift in gaming around the time of 2nd ed that redefined the hobby away from the "game" and toward "storytelling". This was, I think, partly due to boredom with the status quo of hack and slash, but also perhaps due to an air of pretension that had infiltrated the industry. If you read the published adventures from the early to mid-nineties, there were lots of creative stories, but they seemed written to be read, not played. It nearly killed the industry.
Thankfully, new creativity entered the marketplace, and a new format for open-ended, player-directed storytelling emerged. Now players are very sensitive about control. But how does a GM juggle creativity and endless possibilities?
The secret, it turns out, is also the secret to improvisation. Here's something actors never tell you: improvisation does NOT mean making everything up on the spot.
Have you ever watched "Who's Line is it, Anyway?". If you haven't, I'll paraphrase. It's an improv comedy sketch show. Comedians take cues from the audience and then launch into remarkable improvized sketches. So, Dana Carvey turns to the audience as says "Name a dead celebrity!?", too which people shout answers. And then Dana hears "Elvis" and he launches into a sketch about Elvis. As he busts into a seemingly off-the-cuff and totally hilarious musical number, everyone in the audience and at home is amazed at his quick wit.
But, here's the catch... Dana didn't make it up "on the spot". Improv is performance slight-of-hand. He knew somebody in the audience would say Elvis and he was listening for it. And his script was written so that he could easily insert Elvis, or Britney or whoever else was a sure bet to be called. There's still a lot of "off-the-cuff" improvisation in the sketch, but the big punchlines have all been worked out between him and his teammates in advance of the performance. The more you do improv, the quicker and better you become at predicting what the audience will say.Given unlimited choices, it's amazing how often people will take the easiest answer.
So, the reason I share this with you is that the GM can learn from master improvisers. If you prep your players with loaded questions (i.e. colourful NPCs with equally colourful motivations) and prepare the big punchlines (major plot points, locations, etc), then everything else falls into place.
It needs to be made clear that your players SHOULD bumble around. In order for the game to work properly, and in order to give your players the control they desire, you have to let them do whatever they think they should do. Don't ever tell them they have to go on a quest. Don't ever condemn their decisions. They have become accustomed to being lead around by their GM. They need to learn self-sufficiency. That may mean they face certain doom, but that's a lesson learned. They'll be smarter next time.
It's amazing how quickly players pick up on cues once they know they are expected to lead the adventure.
As for preparing when you don't know what your players are going to do... bare this in mind: if you don't railroad them forward, the players will progress slower. A 5 hour session may not result in much actual task completion. If the PCs are in a dungeon, they may only explore 3 or 4 rooms. If the PCs are in town, they might not leave the bar. But that does not mean the session was boring. It means the session was emmersive, complex. My point here being that you will have time to prepare and revise between sessions. You don't need your whole campaign mapped out at the beginning of the adventure. In fact, that's a terrible waste of time. If the PCs do get beyond your prep, introduce a random encounter. The good GM can skillfully weave random encounters into any situation and make it feel intentional. And it prevents the PCs from getting too far ahead of you.
That's my advice. Don't wear it out.