That isn't the rule though.

Surprise is applied per-combatant, not per-side.
Surprise is Dex Stealth check per hiding foe vs. passive perception of all creatures being surprised.
This rule as written typically fails about 90+% of the time for a 5 PC party. As a general rule of thumb (since foes tend to have similar passive perceptions), 10% of the time everyone on the opposing side is surprised, 0% of the time some of the opposing side is surprised (some is not), and 90% of the time nobody on the opposing side is surprised. So, not only is it rare that anyone is surprised, but it's an all or nothing thing which obviously does not make a lot of sense. So your point about it being per combatant means little.
Simple example. 5 PCs. None of them are wearing heavy armor (to make the example simple, but obviously heavy armor makes stealth even more difficult). All of the PCs took the stealth skill (again, to make the example simple, but obviously this does not always happen at every table) and everyone has Dex 14 (again, doesn't always happen).
5 PCs with +4 to Stealth vs. a group of goblins with passive perception 9. All 5 players have to roll a 5 on the dice or higher. The percentage chance of doing that are 32.768%. So, 2 times in 3, the PCs fail at this. In a party of fairly stealthy PCs against dumb creatures that are the opposite of perceptive.
Now 1 in 3 sounds like good odds, but I stacked the deck there. Very few tables have parties that are this stealthy and are fighting foes that are this dense.
Let's change it to a more typical group of 4th level PCs against the same goblins. 2 PCs are in heavy armor and have Dex 12 (one trained in stealth to offset disadvantage a little, one not trained), 1 trained PC has Dex 18, 1 trained PC with Dex 14 and 1 untrained PC with Dex 14 (heavy Dex party, 3 PCs trained). The foes are a group of Drow with passive perception 12.
The chances are: 56.25% * 42.25% * 90% * 80% * 70% = 12%.
Now let's use a group of Drow with passive perception 12 (a mere 3 points higher): 36% * 25% * 75% * 65% * 55% = 2.4%. One time in 41, it's worth rolling the dice.
Why is the table even rolling dice at all for something that will happen 2.4% to 12% of the time? The rules are practically not usable. The players will give up on stealth as a general rule because it works so infrequently.
And in these two examples, the foes have the same passive perception, so it's an all or nothing thing.
Yes, the rules here are simple, but they just aren't very fun. Hence the reason I changed them. Using the group stealth rules with passive perception means that the party will succeed more often (again, still all or nothing with these examples) and using group stealth with active perception rolls means that the party is successful a lot, but only for some of the foes. Some of the foes are not surprised, some are.
The same applies for NPCs trying to surprise PCs. The game is a lot more fun if some of the PCs can react to the ambushing monsters than if the entire party is just sitting there taking it on the chin for at least a full round before anyone can act.
So, your per-combatant point (for the stealthy side) is true, but it's not very true for the other side where as a general rule (i.e. a very high percentage of the time), everyone is surprised or nobody is.
The stealth rules as written are basically only good with a very small PC party (3 or less) or when the DM sends a very small ambush against the PCs (again, 3 or less foes).