Using the food analogy, what if half your group loves nuts, while the other half are allergic/don't like or what have you in regards to nuts.
Nuts are a single type of ingredient in a dish. Avoiding nuts and still pleasing everyone is easy enough. I know, because I actually live that one too - in another group I have a player who doesn't like cheese. And my wife is deathly allergic to strawberries. I know well how to compensate for single-ingredient issues.
Not liking a single ingredient is not similar to having a drastically reduced overall diet. Think, instead, about the person who is a raw foods vegan, or who keeps strictly Kosher, and you're more on target for what I mean.
My friend who doesn't like food - I have no problem with him, as he's not complaining about how nobody ever cooks for him. He recognizes he's a bit peculiar, and manages. And in return we do occasionally try to meet his needs - find the one kind of cookie he likes, or do simple burgers for dinner. The rest of us would be happy if he'd broaden his palate, but we're okay with him. We'd be a lot less okay with him if he tried to insist or repeatedly insinuate that the rest of us had to change for him.
I'll make it really simple, sans analogy - you like only one game? That restriction is inside you, not outside in the rest of the world. It is your challenge. There's only so much the rest of the world can be expected to do to accommodate you. The world is not going to sit still and not look for new things because your tastes are inflexible. So long as you cannot find a way to have fun doing other things, you're going to have that problem, as those without your peculiarity will want to spend time doing other things, and won't be available to do your one thing.
Edit: Seeing Dannager below me, I should note: I don't think of having narrow tastes as a personal flaw. Expecting that somehow the rest of the world should flex to you, rather than you to it, is perhaps a flaw.