I don't mind racial penalties. Just have them make sense. For example, elves are supposed to be some of the best wizards around, but they get a +2 dex and a -2 con. They get a +1 to AC and ranged attacks, but at the expense of 20 HP by level 20? And their favored class sports a d4 for HP?! WTF?!
This is one of those areas where the changes to the core system by the 3e designers had some unforeseen consequences when combined with traditional elements of D&D. The racial modifiers for elves make sense when you consider the concept that is being modeled: they are more graceful but less hardy than humans.
I don't think that D&D has ever described elves as being the best wizards. Elves have always been described as being fascinated by magic and as having an aptitude for the magical arts. Magic comes naturally to them, but that isn't the same as saying that they make the most powerful wizards.
In early versions of D&D, this natural talent for magic was demonstrated by the fact that elves (and half-elves) were the only non-human PC race that could advance as magic-users. In addition, while a human wizard had to dedicate his entire life to the study of magic, an elf could be a competent wizard as well as an accomplished swordsman. However, elves were never able to become the most powerful wizards; in AD&D, they were capped at 11th level, 10th level in B/X D&D.
The 3e concept of "favored class" was not intended to indicate what class a race is "the best at." The favored class is something that the race is naturally drawn to and/or has a natural aptitude for. In the case of elves, it means that all elves can dabble in magic use without hindering their progress in other pursuits. In game terms, an elf can take a couple of levels of wizard without getting hit with an XP penalty for having class levels too spread out.
Unfortunately, of the core 3e races, elves stick out as not really having any features or ability score adjustments that enhance their performance in their favored class. The other core races have better support for their favored classes, but it seems to be because their racial features in AD&D just happened to work that way.
In 3e, any race can be a wizard, and any race can combine being a wizard with virtually anything else, so elves no longer have this advantage. Intelligence determined the chance that a magic-user had of learning a spell as well as the maximum number of spells per level that he could learn, but it did not grant additional spells per day, nor did it increase the power of their spells. Also, the maximum spell level that a magic-user could cast was determined by an Intelligence score of twice the spell level (so 5th level with a 10 Int, 9th level with an 18 Int), instead of 10 + spell level. Thus, not having a bonus to Intelligence wasn't that big a hindrance to being a magic-user.
So we could make elves into better wizards by giving them a bonus to Intelligence, but that's a pretty clumsy solution. Being able to pick up a couple of wizard levels without incurring an XP penality isn't particularly useful, due to how 3e scales at higher levels. I understand how we got to this point (elves were talented wizards by virtue of being one of the few races able to be wizards until 3e changed that), but I don't know what the solution is.
I could go on a tangent on how D&D has never been able to figure out if elves are the woodsy xenophobes or metro spell-slingers and how 4e finally separated the two types into elves and eladrin, but that's a rant for a different thread.
Without going too far off on than tangent, I would say that B/X D&D and AD&D 1e portrayed elves as reclusive, woodsy spell-slingers pretty well, even before AD&D started adding PC sub-races to muddy the waters.