I don't think they make that big of a difference, because players aren't dumb. People don't play race/class combinations where the penalties make much of a difference, and when they do, they buy the stat up high enough (or put a large enough die roll in the stat) such that the character is way above average at that thing anyway. Once the character's actually together, it doesn't matter that Grumblebunch the Dwarf Bard has a racial charisma penalty, because chances are he's going to be one of the most (likely THE most) charismatic person in the party regardless.
The primary impacts of penalties are aesthetic - and a lot of people very reasonable enjoy the aesthetics of ability score penalties - and making certain race/class combinations less appealing. They don't do a good job at all of making races feel any different in play, because a -1 on a d20 roll is essentially imperceptible to begin with.