When you are bringing in limits and denying that something can be done or allowed, the burden of proof is on you. "It isn't contradicted by the text" isn't enough. "It is reasonable" isn't enough...and, as far as I'm concerned, it isn't reasonable. A typical town guard is like level 2ish, and can survive all but the worst 20' falls with no serious injuries, and most 30' falls, since they have 11 HP. As long as there's someone at the bottom to stabilize them, they very frequently survive a full 50' fall and be right as rain the next morning.
That's objective. Baked straight into the rules. Falling is 1d6 per 10'. Town guards have 2d8+2 HP, static 11. Instant death only occurs at -100%, which would be 22 total HP. 5d6>21 only 15.2% of the time. Even a 60' fall is still more survivable than not (45.36% chance of instant death.) IRL humans, even in safety equipment, die from falls as short as six feet, and have fatality rates on the order of 30% or more from falls as short as 20'.