And I tend to find that (ignoring a plot completely and moving in a different direction) pretty annoying unless I get some heads-up that such a move will be coming into play.
Annoying? Why?
Why is it more important that what the DM wants the PCs to do is more important than what the players want the PCs to do?
It's not the DM's PCs. It's the players PCs.
Granted, constantly doing this would get old real fast, but it should be rare enough that a DM could be flexible about it.
But if we're in the middle of a story, and then decide "Hey, I can turn invisible and fly! Let's drop this and go rob people in Major City X!," then I get perturbed. I'm fully aware that "Don't play with those kind of players" is the best advice, but even good players can have a bad night and flake off.
Well, I suspect that most players tend to not do this in the middle of a story. I cannot recall seeing it in a recent game. They might go off on a wild goose chase, but that's ok as long as they players think that they are staying within the story.
I cannot recall a situation where the players just said "Screw it. The Mayor is being a pain in the butt. Let's blow this town and move on.", but over the decades, it probably has happened in one way or another in some of our games.
I could easily see it happening though and as a DM, I would just let that story arc fall by the wayside. I wouldn't get perturbed with the players and I definitely wouldn't try to force them to play that storyline or punish them for quitting mid-story. I created a cool cave complex for them to explore and they didn't go there? Oh well.
It happened early on in one of the WotC podcasts. The DM even got to the point where he drew the map with (IIRC) sarcophaguses on it and the players said "Hmmmm. That looks scary. Let's not go in there." and they walked away.
That might annoy you, but I would just blow it off as the players making their own decisions for their PCs, regardless of what specific thing I prepared.
Sometimes, it's better to just let the players decide instead of deciding for them.
Your "warrior in a well" example is telling. Why does that situation require water breathing or levitate? Why aren't you tying a rope to the halfling and throwing him in after him?
Yes, it is telling. You're assuming that the Fighter wants to come back out right away, even in the middle of a combat.
The fighter falls in and the players decides to explore the real deep water of the well before coming back. Maybe the encounter is in mop up mode up above (assuming the PCs are in an encounter) or whatever.
With the current rules, he's quasi-prevented or at least discouraged in one way or another from doing that.
That's what I am opposed to. The fact that as a player, I say:
Fighter: "Since I fell in anyway, I'm going to explore this deep underwater well a bit."
And the response from the DM is:
"The well is only 15 feet deep. You find nothing."
or
"The well is really deep. Give me Endurance checks as you sink down further and further."
I want the ability as a player to sometimes have the option to control this situation to some extent. If I have a Potion of Water Breathing, I can go down into the water and explore without worrying about the pesky Endurance checks.
Without the option of such a potion, I'm stuck with a bunch of crap like:
1) Get group consensus that they want to allow the Fighter to split from the party (and hope the DM isn't one of those who bought into the whole "never split the party and punish players for trying it" schtick).
2) Get the party Wizard to cast Water Breathing on you via ritual. Ok, first I have to convince him, or possibly even negotiate something with him to get him to do so. Then, it takes 10 minute of in game time to cast the ritual.
3) The Fighter could even be in a situation where getting back is difficult, so he wants to explore first, then head back and he cannot do so because the Wizard is needed to cast the ritual.
There is a certain amount of "this is how the game should be played" and not doing so is bad/wrong fun from our gaming community at whole precisely because the designers put in game design roadblocks to certain options.
If someone wants to breath underwater, it takes 10 minutes and requires the PCs to include a ritual caster in their group. Why? Because someone thought of a cool idea called rituals and didn't take into account the possibility that the game might actually play smoother and have some more awesomeness to it if the players could just decide that the PC does something immediately because he prepared ahead of time instead of waiting 10 minutes of in game time.
I'm not opposed to rituals. I'm opposed to rituals being the only way to do certain relatively harmless game actions that should be allowable quickly and easily.
And since 4E came out, I can count the number of rituals used in my various games on two hands. They are hardly ever used as they are currently implemented in our games. That will vary from group to group, but they almost seem like a wart on the system for me. They aren't a cool way to get things done, they are almost an afterthought where the vast majority of time if they are even considered an option, a player says "Do we have xzy ritual?", the answer is often no. Ok, let's try something else.