I often considering using the longer rest times for wilderness travel and switching back to the shorter ones for the dungeon crawl. I know it is a metagame element, but I hate the tendency to nova in every wilderness encounter simply because the probability of more than one or two encounters is very low.
I also have been using a hack of the TOR/AiME Journey rules, which helps alleviate the same problem.
As this seems relevant for you, here is my version of gritty realism (as refined over a 50 session, 2 year long campaign and numerous forum debates). Note how rests run into each other and the quick rest type (breather). I added those after playing gritty realism for about a year. Players found the tweaks very welcome because it often came up that they wanted to shift gears due to plot events, and HD remained a meaningful limit. Crucial is the 1:3 ratio short to long rests. Otherwise your warlocks and battlemasters make your long rest class characters feel bad. (Warlock slots refresh too often relative to wizard slots for e.g.).
Breather
A breather is a period of downtime, at least
1 hour long, during which a character performs no more than lowkey activity such as reading, talking, eating, drinking or standing watch. If it is interrupted by adventuring activity—fighting, casting spells, marching, or similar—characters must start the rest over to gain any benefit from it.
At the end of a breather, characters can spend Hit Dice to regain hit points.
Short Rest
A breather in which characters go on to sleep or trance may be extended into a short rest of about
a day. At the end of that rest, characters who prepare spells can change their lists, and features that can refresh at the end of a short rest, do so. Those who rest in comfort and eat and drink, recover one level of exhaustion.
Long Rest
A short rest can be extended into a long rest of around
three days. A character must have at least 1 hit point at the start of such a rest to gain its benefits, and must sleep or trance each day.
At the end of that rest, characters regain all lost hit points, and they regain spent Hit Dice up to half their total number of them (at least 1); any features they have that can refresh at the end of a long rest, do so. Those who rest in comfort and eat and drink, recover completely from exhaustion.
Between Rests
When characters finish a rest incorporating a given type, they can’t benefit from another rest of that type until time equal to its duration has passed, e.g. characters finishing a short rest can’t benefit from a breather for an hour.
Sleeping and Trancing, and Armor
Characters who sleep need 8 hours to do so, while those who trance need only 4. Warlocks benefiting from Aspect of the Moon can spend 4 hours reading their Book of Shadows instead of sleeping.
Characters sleeping or trancing in medium or heavy armor aren’t comfortable. At the end of an uncomfortable rest, characters don’t reduce exhaustion and regain a quarter of their spent Hit Dice instead of half.