T. Foster said:
The source literature. Conan, Fafhrd, Kothar, Thongor -- seemingly all of the "barbarian" characters of swords & sorcery fiction had some sort of an innate ability to detect "witchcraft" and "unnatural" things, so as to better fight/destroy them.
You're looking at it wrong to say that the AD&D barbarian doesn't use magic because he's "afraid" of it -- he doesn't use it because he hates it, it's unnatural, and unmanly, and ultimately destroys all those who rely on it too heavily. And unlike weak city-dwellers the barbarian doesn't need magic to succeed, because he gets naturally what other characters need to use magic to get (extra hp, better AC, better saves, faster move rate, faster healing rate, ability to jump, climb, and hide, ability to detect ambushes and sneak attacks, ability to damage "enchanted" creatures, ability to see through illusions, etc.). I don't have the books in front of me to dig through for quotes, but I'm almost certain there are passages in Gardner Fox's Kothar books (which are, IMO, as probably closest in feel to "D&D" of anything in the pre-D&D canon) that say more or less exactly that.