LostSoul
Adventurer
If it worked the way you imagine, heavy weapons would lose their only advantage, rendering them useless, and everybody in history would have used the fastest, tiniest skunkstabber they could find to fight each other.
It does work the way I imagine because that is the system I use in my game.
Let's say you have a dagger and you're fighting a guy with a greatsword. The engagement has just begun so you're 10' apart. If you want to do any serious damage you need to get in close and stab at a vital spot. So you say, "I get in close and stab him in the neck." That's fine, I'd say that's a "Deadly" strike (assuming there's no armour). The guy with the greatsword has more options. He could slash at you or try to run you through, both of which are "Deadly" attacks (again, assuming there's no armour).
That means the guy with the greatsword is going to get some bonuses to his attack roll and AC.
Now if the guy had a sword and shield, I might say that it's not even possible to stab him in the neck - you'd need to do something to get around his guard. Same thing with a dragon - if you want to stab it in the eye, you will need to climb on its back or something like that. But if you do get in close - right up in his face - your dagger is going to be more deadly than the greatsword or sword and shield combo. He'll have to push you back before he can use his weapon again.
I think all of this is abstracted out in D&D combat - you don't know what the characters are doing until the dice have been rolled. In that sort of system damage dice work well, for the reasons you give.