This is a different and unlike the above concern, a real issue of the adventuring day. That's why if you want to preserve the game balance of party vs. creatures AND the game balance between classes(spellcasters and martials), you should move to the alternative rest rules and only have one long rest per week. It's still narratively nonsensical, but it's the lesser evil between 6-8 encounters in a 24 period, unbalancing the game in favor of certain classes and the PCs in general and having a balanced game where rests don't make sense narratively.