It's always acceptable regardless of the situation, unless the DM has somehow married the campaign's continuance to the finding of said BBEG (in which case sticking said BBEG behind a hard-to-find secret door might not have been the best option).
Yes, and this can be done immediately, or at some later time such as while the party's away on its next adventure, or never...all depending on enough variables that there's no overarching right answer.
It's important that someone find a way to salvage these things but why does it have to be the DM?
Why on earth would they call it a night? The players via their PCs can be proactive in seeking out a new adventure (at which point the DM can and reasonably should step up and provide something), or they can engage in some downtime activities including dividing such treasure as they managed to find (at which point the DM has the option of having some sort of adventure come calling), etc.
Lan-"mentioning [MENTION=42582]pemerton[/MENTION] here to get his eyes on this thread, as he's often gone on about fail-forward in the past"-efan
To your last question, the answer is: Because not everybody runs their game that way.
My group tends to have the GM make a custom adventure based on the last session and player feedback on what their PC was going to next. The players tacitly agree to pursue the plothook for that session. Which following the example, means this session was "hunt the BBEG down in his lair and kill him" because it's obviously based on what we wanted to do last week, and we know there's no material for going to the Southlands just for the heck of it.
If the GM makes a mistake in planning that out, and the whole thing hinges on a search check that fails, the party may get stumped for what to do next.
In my version of the example, the GM has some culpability. And it's easy enough once he realizes it has actually happened, to make an adjustment.
If the players are still working through alternative solutions, there's no need for the GM to apply an FF, the PCs are already doing their own.