Just as there is no reason to assume that the "magical handwaving" explaination of imaginary beings interbreeding is true, there is also no reason to assume that they are the same species, genus or have any other biological relationship to one another whatsoever. Either way of explaining it is a matter of personal choice, and the amount of suspension of disbelief the group can handle. In a great many games, (not mine, but many nonetheless) science as we know it doesn't even work, let alone rule the world. A system that operates with parellels to our own concept of physics, biology etc, seems to be in place in these games, with notable exceptions; magic works, allowing for all kinds of gross violations of causality, thermodynamics, and don't get me started. Gunpowder doesn't work. Who cares if you mix saltpeter, sulfur, and charcoal in the perfect proprtions? You get zip. This speaks to the nature of the difference. A fantasy world does not, and some would say, should not model our own reality.
Personally I prefer to think that an elf's "biology" is as different from our own biololgy as a virus, with some convergent parellels (similar organs etc). Form follows function. In my games, an elf (or any nonhuman race) will never be simply another type of human, and I and my players like it that way. To me, saying a dwarf is hardy is like saying an eagle has good eyes. Similarly, saying an orc is bloodthirsty is like saying rattlesnakes seek out warmth. A simple fact based upon knowledge of the beast in question. How can that be racist?
Parenthetically; I'll say that I too have concerns about the semblance of racism in our hobby. I've had this discussion with close friends, and have even taken the side that the game does have those undertones, but I have come to think that a mere discussion of the differences between people/beings/creatures does not constitute racist thought. Even when those differences are that the (fantasy)"race"/species in question eats humans and is therefore hated by most of said humans. Many humans feel the same way about any predator, be it a wolf or bacterium. Others can see past that and recognise the natural order of things.