I can get past dragons (and flumphs) because they are so limited and a world like FR has an enormous portion that's still remote and unexplored. There are also planes of existence for other creatures to live on, that could rarely come into contact with the Material plane.
My thought is maybe I didn't explain it well enough. It just seems to me that all these races with cultures, especially since they live together, seems like they would clash a lot more. Where as a dragon, even an evil one, might realize that they need men, dwarves, etc. to be fulfilled.
Here is a small list: Aarakocra, Azer, Bugbears, Bullywugs, Centaur, Chuul, Cyclops, Drow, Duergar. This is just from the MM and only using A-D. I did not include other planar creatures, as for some reason it's easier for me to rationalize. I only included creatures (races) that are large in population, carry culture, and have contact with the traditional fantasy races. I don't know, it seems like a lot to me.
I should also point out that I like D&D settings. It's just from a world building perspective, this one thing seems "off" to me.