I know it's cliche by this point, but... page 42 is what fills in the gaps, for me. Rituals handle things out-of-combat, but for creative spell use in combat, page 42 is my starting point on how to resolve it. If someone wants to use Ray of Frost to freeze the pool of water their enemy is standing it, I might require an Arcana check in addition to the attack roll, and success might let the attack immobilize the enemy instead of slowing them. I've seen character's use Ghost Sound to draw away the enemies attention and gain combat advantage, things like that.
I know that some might not feel this is the same thing - it relies on the DM deciding how things works, and while the guidelines help with that, they are only a starting point. And it may ultimately feel arbitrary compared to coming up with the creative uses that one knows are going to work within the rules. But... it is still a solid way to resolve creative ideas in combat, and typically results in creative actions that help the combat but don't instantly end it.
In many ways, I find that more enjoyable than finding some obscure loophole between two powers that vaporizes an enemy on the spot.
In terms of just using non-blasty powers in creative ways to control the battlefield (as in the example of using Windwall, Teleport, Invisibility)... yeah, I think people are vastly underestimating utility powers. Controlling the battlefield is what the wizard does, and stunning or blasting groups of enemies isn't his only option. One can totally set things up to break up enemies with cleverly placed walls, leave decoys to draw enemies to the wrong places, turn allies invisible, set up arcane gates to teleport them to where they can do the most good. Walls, illusions, teleports - these all still exist, and a good wizard can completely alter the course of battle without ever blasting enemies with a fireball.
Finally, in terms of certain 'plots' being out of the question, such as "teleport into the heart of the enemy stronghold, rescue the princess, book it back to safety"... if the DM is interested in such a plot, they can easily make it happen. They can come up with a ritual or set up NPCs that can allow the party to make their teleport rescue.
If the DM isn't interested, then they can't - as opposed to the DM having the party cleverly use teleport to bypass his entire adventure. Or thus having to come up in advance with anti-teleport magic to prevent this. And then having the party come up with anti-anti-teleport magic to let them go ahead anyway. And back and forth, and so on, and...
Yeah, I know that some enjoy that level of competition between the players and DM, and figuring out exactly the right tool to ensure they have the upper hand. It could be a lot of fun. But it could also be downright poisonous, and I can understand why they wanted to get away from that style of gaming. As it is, the same exact plots are available if they are the game the players want to play, and the DM wants to run.
It's easy to look at 4E and feel like there aren't those opportunities there anymore. Before, the DM could set things out like a puzzle - "the princess is locked in a tower, how will you get in to rescue her?"
And the party could climb the walls, or break in through the front door and fight their way up to the top, or fly in on a magic carpet and break her out through the roof, or teleport in, or sneak past the guards invisibly, or pop in through the astral plane, or summon allies to break the princess out, or try to trick the guards into moving the princess and then rescue her along the way... or any number of other things I can't think of.
But... you can still do most of those in 4E. Maybe not at level one, but by paragon, most of those seem viable - you can climb walls, you can get flying mounts or items, you have teleportations that will let you breach the walls, you can sneak, you can use invisibility, you can distract or deceive the guards, etc. There are rituals that could help, there are utility powers that could be used for polymorphing, filling the halls with fog, or any number of other things. You probably wouldn't be running things in combat rounds (if not actually fighting), and might instead be using a skill challenge as part of this, or could use a blend of skill challenge and combat (as the party sneaks around, and occasionally needs to ambush some guard minions before they can raise the alarm.) There are any number of approaches you could take - just like before.
There might be some previous options you don't have now. There might be some new ones you didn't have then. Odds are good that regardless of what your options are, it will be a significant undertaking - in some ways, I prefer that to "I cast Teleport. We grab the Princess. I cast Quickened Teleport."
Anyway. Skills, stunts, rituals, utility powers. That really does open up a lot of options in the system. Yes, it can be easy to fall into the trap of never looking past the character sheet. But the options are there for those who want to use them.