But the HP rules say nothing at all about pain. They just say you regain HP after a night's rest, so you're no longer close to death. It would be totally compatible with RAW to ask your players to track the amount of HP damage taken over the past week (completely ignoring healing) and use that as a guide to how much pain a PC is in right now.
If a centaur guts you with his lance for 30 points of damage, and then you fall off a 30' bridge for another 10 points of damage, a night's rest may give you back all your HP, but it's perfectly fair and consistent with RAW to say that your gut still hurts a few days later. And it might leave a scar.
NB: I don't want to hijack this to muse on the intent of HP, but since we've gone there...
I think you just hit on the head an issue which could use some official clarification. In most cases, I don't think HP signify real wounds. Rather, they signify fatigue. The more fatigue you suffer, the more sloppy your guard. The more sloppy your guard, the more likely you are to be struck a blow that will take you out of the fight. When you, as a Fighter, are battling a hobgoblin, your longsword against his morningstar, when you "hit" each other for "X HP of damage," it's not a slash with the sword or bash with the morningstar that causes a wound. It's a simulation of his - and your - mounting fatigue as you try to kill your foe and avoid being killed by him. You're frantically dodging (not Dodge per the rules, but the normal shifting of position in the fight) and parrying and feinting and everything involved in a duel to the death. When he "hits" you, your adrenaline spikes. Your muscles go into overdrive when you desperately fling your shield to interpose it between your head and his spiky thing, the shattering THUD that travels up your arm when you successfully block it. When you "hit" him for enough HP to reduce him to zero, it's a numerical simulation of his fatigue causing his guard to drop to the point that you find the perfect angle of attack to stab him in his throat.
It's easy for me to envision this thanks to my background in HEMA. I can attest that, in a fight, I can never once be struck by my opponent - not even a love tap - but still be struck a killing blow once i get so fatigued my guard drops. One mistake = shish-ka-Bob. [insert rimshot right there, if you please. I think I earned that one.]
That's pretty easy to envision. Harder is the simulation-of-fatigue for things like
spike growth and
fireball and other effects. There I suspect it's to simulate the bumps and scrapes you experience that add up to a Pile of Ouch which causes your fatigue to mount.
Thunderwave throws you across the room, and sore muscles and bruises make for sloppy defense. Scrapes and punctures from trying to get through whacking great thorny bushes is distracting, which affects the attention you can devote to your guard.
Of course, there's always the certain circumstances which cause you to suffer enough damage all at once to snuff out your life, like a whole bunch of
lightning bolt damage. I recommend envisioning a convicted murderer getting the full effect of the electric chair. He's healthy, at "full HP," as it were. But he simply gets hit with so much juice all at once that it fries him to a crisp.
In-game Rests are much easier to understand using this visualization. After a good night's sleep - a Long Rest - your fatigue is gone. You can face the next day's fights with a clear head and rested muscles. A Short Rest lets you tend to those niggling aches and pains, those scrapes and bumps, and take a bit of a breather to rest tired muscles. You can have a snack to replenish your body's stores of energy, maybe ten minutes of naptime.
Powers like the Fighter's Action Surge are also more easily explained. Fighters have learned to draw upon a reserve of energy and concentration which allow them to attack when otherwise they couldn't.
TL;DR - Envisioning HP as simulations of the rigors of combat makes wrapping your head around "a breather (e.g., a short rest) makes it a bit better" and "a good night's sleep makes it all better."