Whether it is 3.5, Pathfinder or 4E all of these systems have fixed damage dice for falling, 1d6 per 10 feet for 3.5/PF or 1d10 per 10 feet for 4E.
It's fine and simple. It assumes that PCs are medium or small creatures and so their mass is close enough that when they fall they take an amount of damage that is fairly comparable. The rules are really intended to see what happens to PCs most of the time, rather than seeing what happens when bigger or small creatures fall.
However, when it comes to weapons then the "physics engine" kicks in a bit more (Force x Mass = Momentum) and suddenly the mass of the weapon becomes important, and so we have die steps keyed to the size of the creature. It's a bit more fussy and detailed, but since the main point of D&D is to conduct combat then this level of granularity is generally desired.
Now if you wanted to extend the physics engine a bit more and didn't mind that extra little bit of information to keep track of it seems rather simple and realistic to key falling damage to die steps based on size. Something like:
3.5/PF
Fine 1
Diminutive 1d2
Tiny 1d3
Small 1d4
Medium 1d6
Large 1d8
Huge 1d10
Gargantuan 1d12
Colossal 2d6
4E
Fine 1d2 (I think they got rid of this size?)
Diminutive 1d3
Tiny 1d4
Small 1d6
Medium 1d10
Large 1d12
Huge 2d6
Gargantuan 2d8
Colossal 3d6
Does this sound right? Or at least not game breaking?
It's fine and simple. It assumes that PCs are medium or small creatures and so their mass is close enough that when they fall they take an amount of damage that is fairly comparable. The rules are really intended to see what happens to PCs most of the time, rather than seeing what happens when bigger or small creatures fall.
However, when it comes to weapons then the "physics engine" kicks in a bit more (Force x Mass = Momentum) and suddenly the mass of the weapon becomes important, and so we have die steps keyed to the size of the creature. It's a bit more fussy and detailed, but since the main point of D&D is to conduct combat then this level of granularity is generally desired.
Now if you wanted to extend the physics engine a bit more and didn't mind that extra little bit of information to keep track of it seems rather simple and realistic to key falling damage to die steps based on size. Something like:
3.5/PF
Fine 1
Diminutive 1d2
Tiny 1d3
Small 1d4
Medium 1d6
Large 1d8
Huge 1d10
Gargantuan 1d12
Colossal 2d6
4E
Fine 1d2 (I think they got rid of this size?)
Diminutive 1d3
Tiny 1d4
Small 1d6
Medium 1d10
Large 1d12
Huge 2d6
Gargantuan 2d8
Colossal 3d6
Does this sound right? Or at least not game breaking?