Your characterization of the fears as "self-imposed" is, I think, rather significantly misrepresenting the way fear works.
If you frequently have the experience that other people react with hostility upon finding something out about you, being afraid that new groups of people may do the same if...