So the question becomes, "If a world thought a certain group would produce an unfair ruler, why wouldn't the world just generally kill everyone of that group as soon as they can be identified?" I guess the follow up would be, "Once you've eliminated everyone from said group, would the world then have to kill off everyone of the next likely group that might produce an unfair ruler?" "And the next?" "And the next?" This is not really a good argument for class balance/homogenization.
this is the rallying cry of anti-hate-mongers everywhere. After you kill off all the dark elves, who's next, the wood elves? How long before it's dwarves and other short people.
I suspect that the people who hate Dark Elves have a pretty strong reason to specifically target that group and are willing to accept burning a few drizz't-clones to nail the spider worshipping freaks.
Actual genocide seems to be enabled by a strong party being able to single out a weaker party for extermination. it seems to require special conditions for it to germinate. You seldom hear of successful "let's kill the powerful people" stories.
Additionally, in normal societies, people are too busy worrying about their own lives to notice other people getting more powerful than them. Heck, most people with power are promoted to power by other people. The power is given to them initially, not taken.
Wizards are not a weak party. They're less likely to let situations remove them from places of power and respect. If a sickness came into town, they'd skillfully deflect such blame to the dirty halflings in the street, rather than let the public build animosity against them and their dark ways.
While some people like the magic is evil, hide the spellcasters trope, I find it unfairly cripples some classes, while leaving it OK to be a Barbarian/Paladin/Monk because they don't use magic.
Plus, as a I said, Power doesn't get Persecuted. Not as a general rule. People fear it, respect it, give authority to it, and cling to it, hoping to get some of the benefits from those with the Power.