I would also argue that there would be a need for some training time to fully get up to speed. This was a feature in first and second edition that made sense, as it would take time for people to practice new skills, become proficient with a weapon, or research new spells. Another option is to allow slower advancement at upper levels, as explained in the DMG.
Travel can also eat up time for many parties, even in a world with teleportation and similar magics. It takes time to explore mountains for ancient ruins or to guide a caravan through a desert.
Also, some challenges that are important to characters can take time and not have a direct experience point reward. For example, political challenges -- such as ferreting out a spy in the king's court, rallying people to a cause, or trying to gain a place in the government -- can take time. So can such tasks as tracking down a missing relative, negotiating a trade agreement, or achieving a personal goal. The heroes may find that their success may have created a time of peace with few challenges. However, the heroes should then find that they have earned the gratitude of many people -- who can keep them busy with other things.
The threats that a high level party might consider commensurate with their skill might be few and far between. They may well find that their actions are limited by their adversaries. In which case, the PCs might decide to enlist adventurers to take out some of the supporters of their foes.
I think role playing challenges can help to not only take care of the issues of characters advancing too fast but help to add a sense of depth to a campaign. If Elric, Conan, Hercules, and other heroes had down time, so should your heroes. (Some young epic level characters might exist here and there, but they should be relatively rare like Alexander the Great. Such characters are notable by their rareness as well as their achievements.)