Well, I don't know any modern-day barbarians, and I'm not at all familiar with REH's Conan, but I don't think it's surprising that a character who is supposed to come from a primitive culture might distrust "civilized" magic. As the barbarian advances in levels, he becomes more "civilized." Till...