Yep, this is the kind of thing I was thinking of when I said I just improvised stuff and the player offers of ideas and the die rolls helped determine what I improvised.
Running that hatchery dungeon, I'm not a DM who has thought of every permutation of every NPC's personality and what they like/hate/believe etc. etc. I'd never waste my time doing that because who knows if/when any of those ideas I spent going through would ever actually come up? But once the game is playing and what the players did resulted in a situation such as what you mentioned above (with the one lone cultist remaining)... THAT is the point where my reactions as the NPC to what the PCs say/do/offer to me will "create" the NPC in the moment. So at that point... a bit of RP followed by a Religion & Persuasion check to "convert" this cultist, and the cultist rolls poorly on his Insight? Then sure! The guy converts! Why not? Maybe now the party has an NPC ally that can feed them information about the cult going forward. Or maybe they go in an entirely different direction than they might have based upon the NPC's information?
All this is stuff I never would have planned for prior to this hatchery encounter, it all comes out of improvisation based off of dice rolls and skill checks. So for me... anyone and everyone who wants to roll checks with their skills will go a long way in determining what happens and the directions the party ends up going.
Yeah, if another DM has very specific scenarios written out and knows going in what the goals are and the approaches that would work best to achieve them... waiting for the players to offer up their movement in that way probably works for their games more effectively. But I'm not one who has things that figured out. I'm just flying by the seat of my pants, and a Nat 20 goes a long way in pointing me in the direction my improv should go.
To be clear, my position isn't that you shouldn't roll. It's that, as a player,
trying to roll isn't a great strategy for success because it means you're undertaking tasks that have an uncertain outcome and a meaningful consequence for failure. And when you're out there boldly confronting deadly perils, that can be a recipe for disaster. Some rolls are unavoidable. But I am going to
try to avoid them or, failing that, try to mitigate the swing on the die.
I still submit, as I did upthread, that this specific example is perfectly achievable with the player not asking to make a roll. The player describes what he or she wants to do. The DM considers it and asks for a roll, using a contest for some reason to come up with a DC. The player succeeds. The outcome in this case was the same. But from the player's perspective, why would I
want to roll in that situation? I'd rather my action just succeed. The meaningful consequence that followed my failure on that roll might not have been good.
As for planned challenges, I think it's good to flesh
some challenges out, but doing so for every interaction with a random NPC seems like a waste of prep time. I also don't think it's a good idea as DM to decide ahead of time which goals and approaches work best. The better thing, in my view, is to just come up with a situation and its moving parts, but not any solutions. That's the player's role and I would hope those solutions don't include asking to make checks.