the rules are an abstraction of reality, and these abstractions are necessary because we want a fun game, not an exacting simulation.
As someone with some sword-fighting experience, et me tell you, the combat rules are almost as ridiculous as falling 500 feet instantly. But I accept that, because very realistic sword-fighting rules would be complex and probably not fun.
We shouldn't put too much weight in those abstractions. 5e has decided that diagonals are 5 feet same as the side of the square. If we decided "this is how things are" - then the moon is a cube, not a sphere. So we shouldn't extrapolate too much from these abstractions.
When the rules don't help us much on how something should work (like this instance, no ruleset is complete!), then we should try to find a solution that follows the rules (that mention of a ready action vs a reaction is important) but also respect what we know about physics, and maintain balance. So it's not easy! but we should try. Discounting physics entirely is unwise.