Barbarians fight each other all the time, and the effects of rage are pretty significant. With the effects of rage as the mechanics describe them, barbarian on barbarian fights are extended slugfests.
All the time? In whose campaign?
How many barbarian on barbarian fights do you think people in a gaming world witness (talking barbarian class vs. barbarian class here, not barbarian tribesmen vs. barbarian tribesmen where the class might often be different or not even exist)?
How about Reckless Attack? That would shorten battles between barbarians.
Barbarians are a bit of a MAD class for unarmored defense. That too might often shorten battles between barbarians due to lower AC. Barbarians do not get heavy armor (although one could dip in another class and wear heavy armor, but then he could not rage anyway). Damage resistance is the game mechanic that makes up for this.
And, who exactly is standing on the sideline of a Barbarian vs. Barbarian fight with a stop watch?
No doubt, high level barbarians have a lot of ways to help survival with advantage on saves and other abilities. It's just not necessarily true with GWM, Reckless Attack, and other game mechanics that a Barbarian on Barbarian fight will be that much longer than a Barbarian on Fighter fight (where the fighter often has higher AC, can have feats like Heavy Armor Master, gets Second Wind, can feint or cast spells, gets more attacks per round at higher levels, etc.).
The point is that this POV is extrapolating character information from metagaming information where such info might not be common knowledge in a given campaign world and there are other balance factors that might limit even a Barbarian PC from having this metagame knowledge. The player knows because the player can read the PHB. The creatures in the world should not necessarily know this type of thing. Both the Barbarian and the Fighter take a long time to kill. That's what the characters should know. How the rules allow them to do that? The players know that.