What happens during those more rounds? More attacks are made. What happens when more attacks are made? More hp is lost.
People tend to forget that fighters and martials DON'T have infinite Hit Points. A 20th level martial tends to have 165 hp (except barbarian). This is one of those things people forget about it seems. An orog deals 10 damage per hit. Let's say you have your fighter with their longsword, making 4 attacks for 4d8+28 or 46 damage. That kills one of them a round, so let's say you fight 10 of them. 10 CR 2's versus one level 20.
They get 18 attacks the first round, 16 the second, 14 the third, 12, 10, 8, 6, 4, 2 for a total of 90 attacks. They have a +6, so let's say you have some fancy +2 plate and have an AC of 22. That means they have a 25% hit rate. 90/4 = 22.5 attacks that will hit for.... 220 damage.
Dead fighter.
How are the orogs all able to get their attacks on the fighter in those early rounds? Did the fighter use their action surges or second wind? And what sub-class is the fighter? Didn't take toughness? Why only 165 HP - since they are a tanky fighter (plate and shield) you'd think they'd have
at least gotten their constitution to 18. But okay, we'll make it only con 16 for some reason, which, using fixed HP, gives them 184 HP. Honestly, this fighter is making some bad choices, but whatever; I guess the exercise is to see how well a poorly planned fighter does.
Okay, just to give the benefit of the doubt, let's assume the orogs surround the fighter - so this is a worst case scenario where the fighter has done zero tactics. Also, we'll make the fighter a champion, the most vanilla of the sub-classes. So the AC will be 23, since they would obviously take defense to go with dueling.
Round 1: Fighter action surges and kills two in the first round, which is fine because only eight could have attacked them anyway. Those 8 do 16 attacks, hitting 20% of the time against AC 23, so that's 32 damage. Fighter is at 152 HP.
Round 2: Fighter action surges again to kill two more and then pops second wind, healing 26, so is at 178 HP. Orogs do 24. Fighter is at 154 HP.
Round 3: Fighter kills one more leaving 5. Orogs do 20. 134.
Round 4: Four Orogs, left, to do 16 damage. Fighter is at 118.
Round 3: Three Orogs, 12 damage. Fighter 106 HP.
Round 4: Two Orogs, who really should be running away. 8 Damage. Fighter 98 HP.
Round 5: Last Orog. Fighter 94 HP.
Round 6: no more Orogs. They managed to get the fighter down to about 50% health. Honestly, though if they ever managed to get the fighter below half health the fighter would have been healing 8 HP/round, even with their pitiful constitution. And if the fighter was able to get any kind of cover, such as fighting from a doorway, they could basically tank infinite orogs.
So the only way those orogs are taking down a level 20 champion is basically if there is an army of them and the fighter is completely surrounded with no where to go. Even then it will take them awhile and a lot of dead Orogs.
Which seems about right, frankly.