Regarding HP and if it represents actual physical damage or not, my personal opinion is simply that it is not a Black or White matter. Thinking that there is no physical damage at all involved is absurd (to be fair, I don't think anyone is actually thinking that way), but so is thinking that it only represents physical damage. And I'll take my 5-6 years of fencing as example.
As you gain experience in fencing, you don't actually get better at taking a hit (harder! harder I say!), you get better at parrying, dodging, distance control and managing your exhaustion level. A new fencer simply has no chance against an experienced one, he'll just never hit him and exhaust himself to death while the experienced one will not even break a sweat.
So, as a fighter gain level in DnD, technically speaking, he should get better and better to dodge and parry a blow, that's how you actually survive since, again technically speaking, a sword hit ca potentially be a mortal blow. But there is no mechanic in game to represent this skill, the actual experience of the fighter (well, except in 4e where you actually add your half level to defense to represent that you are getting better to dodge and parry attacks, which makes total sense). If I take ADnD2 as example, a level 10 fighter with no armor will be just as easy to hit than a level 1 fighter with no armor, exactly the same AC. Now, by experience, the level 10 fighter should be harder to hit, because he should be better to dodge and parry. But no, since they both have AC10, they are just as easy to hit. The only difference is that one hit of a longsword can actually kill the level 1 fighter while it will be just a scratch for the level 10 fighter. But how so? A hit should be a hit. Why does it kill the level 1 fighter but is just a scratch for the level 10 one? There must be something else at play linked to fighting experience. So, not only does Hit Points represents physical damage, but it also represents actual skills to deflect blows and the ability to change a mortal blow to just a scratch.
But what about when a scratch is not really an option? Let's say our two fighters are fighting a giant swinging a tree like a mace as a weapon, something that would kill anyone with just one hit. The giant takes a swing at the level 1 fighter, hit him and do 24 damages, killing him right away. The giant takes the exact same swing at the level 10 fighter, hit him, do 24 damages, but the fighter is still on his legs as if nothing happened. So what happened? If he got hit he should now be pudding on the ground... unless he actually dodged the blow. But if he dodged, it means he was not hit, so how come he still took damage. A simple explanation is that dodging this killing blow was actually exhausting. And back to my experience from my years of fencing, exhaustion do play a big role in a fight. The more you fight, the more you get tired, and the more exhausted you are, the more difficult it is to dodge or parry a blow, because you get more and more slower, your reflex is just not there. But again, there is no mechanic in DnD that represent this level of exhaustion; fight for 20 seconds or 5 minutes and your character will still be in top shape... except probably for his HP level that will get lower and lower the longer he fight as he try to dodge and parry blow, until the fatal blow. So it is easy to tie HP to stamina level, as it is the only mechanic that can actually represent this reality (Well, except in 4e with Healing Surges, and I guess 5e with Hit Dice)
So, in my opinion, Hit point not only represent physical damage, but also the skills of the fighter and his stamina, and game mechanics support it. When you accept that it is a little bit of everything, it all makes sense. It explains why a level 10 fighter can take 10+ sword hits and live while a level 1 will be dead after 1-3 hits. It explains how a second wind work as the fighter is taking a moment to catch his breath. It explains why some encouragement words can give you the will to continue fighting instead. Why just a little rest can give you back all your HP. It also explain the value of healing surge because, again from my experience in fencing, everybody as a limit at one point where he just don't have any energy left, where he need to take a long rest to fully recover.
But at the end of the day, DnD is just a game. It is not meant to simulate real life. There is abstractions that is only there to serve the game, that you need to do some mental gymnastic to get around. HP is one of those abstraction that you need to adjust to fit your own personal narrative in this game of make belief.