Wanting a drink? I have no issue with getting a drink in a black gay bar. Do you know why? Because as long as I can get a drink, the bar is serving the function I entered for - getting a drink. Unless they become fractious over my presence, I would feel just fine.
Okay. I suspect many would take that as a questionable statement, but, for our discussion, let us take that as 100% true.
Then you are apt to have a major issue at this juncture, as by your statement you probably lack full understanding of what is driving this discussion. Quite frankly, if that's the case, you may well lack the perspective to understand the issues here.
Please allow me to explain how most of the stated demographic would likely feel - decidedly uncomfortable. Humans, speaking generally, are a highly tribal species. We have pretty strong notions of "in-group" and "out-group", or Them and Us. And we act accordingly. We tend to avoid places that are filled with Them, and not with Us. When put in such situations, humans typically end up with some level of anxiety - and I mean that in a rather clinical sense, including the raised blood pressure and increased levels of adrenaline and other hormones related to stress in the bloodstream, and so on.
This is entirely natural thing, and not the result of a conscious, rational process. It is how most of us are built, an aspect of human nature that we must accept and work with. In order to understand the conversation, you must accept that this happens. Maybe you, personally, do not respond to such matters, but the rest of the population struggles with this every day. Our cultures spend a great deal of time reinforcing ideas of Them and Us, and wanting to be part of Us, and not with Them, drives much of human behavior on the individual and the cultural levels.
The response, "Well, everyone else should be just like me - 100% rational and not caring at all that they are surrounded by Them," is about the same as the response, "Well, they should just suddenly have olfactory senses that are 1000x stronger." It runs contrary to their nature, and they can't just stop being that way. It is, frankly, not workable, a non-starter.