But a 250 HP human who does 40 damage a swing? A human who punches like a storm giant and can take more damage than one.
...It's easy to handwave it with magic. And that doesn't really work.
HP isn't just about how many hits something can take. A storm giant could take way more physical punishment then even the best of non-magical human warriors, and it's ridiculous to think otherwise.
But a CR 12 Warrior Antagonist? He has inner reserves of resolve that focus his mind. He has fought and killed and conquered hundreds of opponents. He has seen (and pulled) most of the dirty tricks a fighter can. He is incredibly adept at judging his opponents and dodging or blocking their blows or spells with honed reflexes. HP isn't just about physical health, it's about fatigue and the will to survive and endurance and anything else that keeps someone on their feet.
So maybe he has as much HP as a storm giant. Maybe when a storm giant takes the hits and loses some HP, he'll laugh off the scratch and swings back, until the giant is streaming blood from a hundred cuts and makes a last rattling roar.
But maybe the Warrior nearly gets hit and the blades come too close for comfort and he starts to grow nervous. His HP drops, but physically he isn't touched. As the fight goes on he realizes maybe he finally met his match. This party, this opponent, is different. And the last strike comes as his confidence finally breaks and he hesitates just enough for Grognard the lvl 11 Fighter to deal the only "hit" that takes his head off.