Yeah, I wouldn't support that as a feature, not the long rest part anyway. Adventures are planned, by a lot of people. around a typical adventuring day of x encounters. Messing about with long rests just leads to power creep and/or some version of the 5 minute workday. Mostly anyway. An ability like the spell catnap, to cut short rests to 5 or 10 minutes, would be a great idea though. A lot of people struggle narratively with short rests anyway, so it would be a well received feature I think.
Edit - Tony's suggestions about limiting LR to once per 24 would fix my issue with it. Missed that bit.