Right now I'm running a 1st-through-20th-level campaign I wrote. We've a party of six full-on spell casters, all of whom are between L11 and L13 at this point. Your proposed solution is exactly what I'm using and I've found it necessary with my party: these guys are so off-the-wall but also so creative in their magic use that at least half the solutions they use are things of which I'd not dreamt. For each dilemma or "puzzle" I give them, I make sure I come up with at least three--usually five or six--distinct, well-defined solutions I can imagine working and how I'll have my NPCs and world react to each in case of success or in case of failure (lotta work, but this is what it takes). What so often happens next is they come up with something completely different, but I manage to react on the fly because I spent so much time thinking through all sorts of possibilities and likely reactions. Then I go back and re-write all sorts of stuff for next week's session. It's heavy work, but we're having a lot of fun.
What I cannot see but someone more experienced might see is how to accomplish all this without a ton of prep. If anyone who's done this a bunch has tips there, I'll be grateful.
I second the "Don't" prep so hard. Don't make detailed solutions. Don't prep more than the next session except for general concepts. Make outlines of the power forces in your world. Give them a non-detailed list of resources (lots of cultists, some politicians... or Unlimited kobolds with some ogre muscle... or numerous merchants with caravans, hired muscle... or numerous cheap taverns and a few warehouses...) and what the motivations of the group are. Probably add a leaders name and description (Bob, a mysterious ogre).
Have a set of generic encounters and locations. Such as a merchant caravan with guards, a group of devils, a strike force of horrors. And then locations, a forest road, a tavern with a hidden basement, a desert keep, etc. Then mix and match those as needed, and add more than one group of NPCs when needed to make the encounter appropriately hard.
Doing all that will give you a set of tools to create a hundred different challenges on the spur of the moment. So when the players go off the rails you have something to throw at them. "Oh, you find a map of the nearby forest with a x marked deep in the forest." Now you can throw a random caravan they can cross while on their way (if they go) and then a hunting lodge infested with devils.
As for the initial challenge you plan for the next session, as has been said, you don't have to work out exact solutions. General ideas of what might be possible is all that's needed. So they could probably use teleport, or perhaps they could summon the solution, or they might be able to planar shift into the closed room, or...
Oh, and don't worrying about balancing the encounters. They party will either figure it out, or run away and come back later. For instance, my party was about level 14 and I used the map from Riverguard Keep (PotA) and put something like 40 devils (chain devils and others) and a couple of BBEG's and placed them about the place (where the watches were setup, were the misc were lurking, etc) and placed some guards and wards and symbol traps and then let the party figure it out. All the encounter guidelines said is should have been a 3 or 4 times deadly encounter. But the party figured it out, and had a blast.
