But I think you're confusing my interest in discussing this topic with some kind of cry for help.
I don't know about a "cry for help" but this thread certainly seems like asking for help to me. There is nothing wrong with that. Many of my posts to Enworld have been asking for help as well.
I'd also add that I have no idea what kind of DM you are; the degree to which I take you seriously will depend on the value of what you have to offer.
I would assume nothing else. However, what I will tell you is that you asked for help, and immediately qualified it by the sort of help you expected to receive. That is, you are asking, "How can I ensure that there are always meaningful consequences of failure to a dice roll?", and there is implicitly to that question the addendum, "...for every fictional situation that might occur." And you are noting that for many fictional situations, finding meaningful consequences is hard and strains your imagination.
The answer you want to get from me is some sort of guidelines for providing meaningful consequences to propositions in the fiction that transcend the fictional state. But what I'm telling you is that you have it backwards. Instead of worrying about the process of resolving the fictional state, instead creating meaningful situations in the fiction. There is no one size fits all answer.
I almost agree with you on this one. There's nothing preventing the players from hiding AND getting their weapons ready (that's what "surprise" is, right?)
Yes, there is! And I can explain it in the terms of your "goal and approach" methodology.
Surprise doesn't involve hiding. It involves the guard not knowing you were there before you acted. So, while there is nothing that prevents the player from hiding and getting their weapons ready, if they phrase this in terms of a goal, then absolutely they cannot both hide and get off a successful ambush. They either can do one or the other.
Suppose they give the goal and approach, "We hide and get out our bows. As soon as whatever is coming down the corridor arrives, we shoot it." Now, they have given a goal and an approach. This is what they want to happen - "shoot the surprised bugbear" - and how they plan to accomplish it is quickly hide to facilitate that plan. But now, things can go wrong. They could fail the stealth check. Depending on how badly they failed it, trying to hide might have been a worse plan than just standing still, so that the Bugbears hears them moving around and so is not surprised. But if the succeed, perhaps they gain additional advantages compared to have standing still. Whatever happens, they are now shooting at whatever comes around the corner, bugbear or not. (Generously, you might allow some sort of test to avoid doing that if it isn't a bugbear).
But now, lets suppose they give a different goal and approach, "We hide and get out our bows." - same approach, but "If we are seen, then we will fire our arrows." Different goal! Different consequences. Now, the issue on the stealth check is whether or not they get seen. If they are, they can fire their bows, but the bugbear knows now that they are there so he's not surprised. The bugbear may also pretend not to see them, forcing a new opposed check. And the good thing is that if it is the princess that comes around the corner, well they aren't blindly firing at her.
All of this naturally comes out of the fiction and the different goals and approaches involved.
So I think a better way of presenting this is by giving the party two options:
1) They can prepare for a regular ambush and potentially get the drop on the monster.
2) They can really try to hide and avoid combat completely, but now if they fail they should be "worse off" than if they had tried #1.
And here we get to the problem of you try to shove your preferred approach on the scenario. You are playing the PC's for the players as soon as you give the party two options. Because there are not only many more than two options available here so that two options is limiting party agency, but honestly I think presenting them options is not actually listening to their goals and approaches. There are times sure when you need to prompt the party to clarify what it is actually doing, especially when the player proposes a mechanic instead of a proposition such as - "I roll stealth." But too much prompting and too much giving options and you might as well be playing a 'Which Way' book.
So maybe one way to resolve it is...and this is what
@Charlaquin was saying...is to let them hide, let the monster approach, and then at the critical moment make them choose which way they are going: do they try to spring their ambush, or wait to see if the monster notices them?
Yes, but ideally you are getting this goal BEFORE the critical moment. If they wait to the critical moment to decide what the goal is, you reasonably should interpret this as chaos as each player decides how to act on their own, potentially thwarting other players plans. And of course any table talk is at that point in character, which is rather harmful to your stealth.
Totally agree about trap design, and I avoid putting un-telegraphed traps into my own adventures. Sometimes it takes new players a while to get used to that, and they still want to "check for traps" on every door and container for a while.
Untraining players of bad habits is a long process. On the other hand, don't discourage players from interacting with the fiction. Just discourage them from primarily interacting with the fiction in uninteractive ways like, "I make an Investigation check." Yes, but what do you actually do?
The real value in what you are calling "goal and approach" - well one of the real values - is that it engages the players with the fiction. But don't let the good results of engaging with the fiction turn you heavy handed in how you run the game. Rather, write engaging fiction that naturally lends itself to interesting and fun scenarios.
