It depends on the players. High level players can be quite efficient the high level PCs I play aren't much slower, although obviously there can be more numbers to track. You just have to pay attention and keep engaged, find efficient ways to resolve your turn like rolling attacks and damage at the same time or even rolling ahead of time.
But others? Once you get to even mid-low levels they have to sit and analyze every possible option just in case there's a more optimal move. If someone is dedicated to rolling one die at a time for absolutely everything then when you have more dice to roll it will take more time.
But I'm not sure you really have that many more options at high levels. Even a 20th level caster doesn't have all that many spells applicable for combat than, say, a 5th level caster.
There are many ways to make high level combat faster. Have color coded dice and roll multiple attacks and damage at the same time. Use average damage (I even allow rounding up) instead of rolling. Accept that you don't have minutes to decide what to do because your PC doesn't either. If someone seems stuck as a DM I'll often list possible options that I see, although of course they can always do something else. As a DM, don't get carried away with overly complex scenarios*.
Some people will just be more efficient at running their PCs at all levels, some DMs will be more efficient at keeping a game moving. Higher level is always going to be at least a little bit slower, in 5E it doesn't have to be significantly slower.
*which can happen at all levels, I just had a combat scenario recently for a level 2 group that took what seemed like it took forever.