It looks like you are really going for realism. I think you are onto something abotu falling speed to damage. However, I think you need to calibrate your damage to height of falls that will kill.
I found a lot of contradictory anecdotal, then I found an article using CDC data.
http://ehstoday.com/nothing-easy-falling-off-ladder
It looks like the 50/50 point of death is about 20 feet. So if you are rolling (either damage or HPs), your average human should die on an average or higher roll and survive on a lower than average roll. (Reverse this if rolling HPs for the commoner instead of damage done to them.)
Maybe a commoner is too weak at 4 HPs, let's go for acolyte or cultist. At 1/4 CR and 9 HPs, that seems to be a common non-leveled but still active person.
So 19 ft/s^2 for a 20 ft drop works out to be about 1/2 a HP per ft/s^2. So take your original chart and chop it in two. (Which isn't too different from d6 per 10' fallen but rising a bit slower.)
This really shows the abstract measure of HPs where two fit and healthy adults could have 50, 100 or more HPs difference. But since there is one case of someone suriving a fall from 33K feet without a parachute, that's where the luck aspect of HPs come in.
Now, part of me feels that you were looking at a more realistic solution because you didn't like the idea of mid-level PCs falling 80' down and dusting themselves off. Unfortunately, and damage level that would seriously impact those PCs would be much higher lethality then in real life. At that point, it's not a more realistic system, just a more deadly one. If you want more deadly, why not just roll more or larger dice?