The most elegant approach (i.e. fewest rules to handle the most circumstances) would be to simply rule that a character can always choose to go prone to stop forced movement anywhere.
This makes a pit or hindering terrain a special case (a reason why they might choose to do so), and it solves the problem of why they can do it in some cases but not in others.
It still leaves open the question of what happens when attacker A forces you to the edge of the cliff (at which point you go prone) and attacker B attempts to force you over (you are already prone, so you can't 'go prone' to avoid the second push). But that is an issue with the current rules as well.
Carl