Re: my Dagger/Letter opener example:
It is extreme, but its not "designed to break the system" but rather to illustrate the difference in weapon proportions.
A longsword sized for a small creature would have a different sized hilt (both length and circumference), blade (length, width, depth & shape), mass & balance than a shortsword sized for a medium creature, even if the 2 weapons are the same overall length.
While that shortsword may be no longer than the halfling's longsword, the shortsword's grip will be bigger around than the longsword's- thus would be harder to wield and easier to disarm. It may even be heavier...or at least feel that way, since the balance would be different. In the other direction, the human might find the shortsword's grip too skinny to hold with surety without using a deathgrip on it- a recipe for fatigue and disarmament just as surely as the other way around.
Could a Warrior overcome such difficulties? Sure- but he's still going to be less proficient with an ill-balanced weapon (one inappropriate for his size and build) than he would be with a normal one. Thus, negative mods.
Look at it this way- Joe Satriani is a great guitarist. He currently uses custom-made Ibanez guitars (valued in the many thousands of $$$) as his main axes. Could he pick up a kid's-scale First Act starter guitar (valued at around $100...with amp) and make it sound good? Sure- but not as easily as one of his chromed-out killers back home. Result: -2 to his perform roll...