Ultimately if you are a DM and you cut the experiences a percentage of the player base wants from your game, you cannot be confused or upset if those players walk. It's your world and your setting and your ability to accommodate those players if you want them.
I agree with this, I just want to use it as a vehicle for explanation.
A table shouldn't reach the point that tension is so great that it's walk-or-not. Sure, kick a bad player, but a DM isn't supposed to be militant because "my word is law!". You're trying to give the players a game to play, so being marginally open to accommodating other people is kind of necessary for everyone to get what they want. If they don't want to play your game, maybe consider your options. If people are actively leaving you, there might be something wrong.
Just because the DM has the final say on rules, doesn't mean that their decisions are unquestionable, infallible scripture. Neither is the PHB, to be fair. Just because the DM makes choices doesn't mean those choices are good for the group (whose wellbeing should be prioritized), etc. They should be open to changing before the walk-or-not player situation as much as the player(s), and there's no good reason that this step has to be skipped. Why must it be an ultimatum? Yes, the DM has power. Yes, the players have a voice that the DM ought to hear. If no-one wants to play Thundercats, don't run Thundercats! Is this not a situation where the DM's power should be confronted, or should everyone just skip to walking?
It's good to have heard some small tales of people trying to launch campaigns with restrictions, with negative player feedback. It kind of proves that the DM can't throw chains on the player without talking it through first, which is ultimately the thread problem.
Restrictions are fine under certain contexts, but there is no fundamental reason that a line has to be drawn at certain points. If you personally don't like a race, that doesn't mean it shouldn't be played in general or even at your table. I will push forward a little- even Worldbuilding isn't infallible, because it can be solved with some more world building. It doesn't matter if it's a class or race, giving players wiggle room isn't hard.