The Drow originally were not merely dark skinned in a human way, but literally black, per Snorri Sturluson's description of the "black elves" (apparently dwarfs, actually, although their dwelling is called Svartálfaheim) as "darker than pitch".
There was a later (2E?) product with a cover depicting female Drow in a very obviously "blaxploitation" style -- even (if memory serves) to the point of lightening their skin color.
The association of darkness with danger or evil has to do with human dependence on vision and consequent vulnerability in the absence of light (as for instance at night or underground). It is not peculiar to pale-skinned people!
Likewise, although (again, literal) whiteness can connote purity, it can also suggest death by association with pale bones. See the stormtroopers in Star Wars for an excellent example -- and the common use in Asia of white for mourning or funereal garb.