I think some people are missing basic human preferences and it gets lumped as racism.
Some people do not want to see a black Zeus or Achilles.
Some people do not like the new take on the Klingons in Star Trek Discovery.
Some people do not want to see a transvestite Superman.
Some people do not want to see the Drow loose their matriarchy status.
Some people did not like Timothy Dalton as a James Bond because maybe he was too real and lacked the charm/grace of the previous Bonds (i.e. he went against cannon)
Some people did not like 4e.
So if someone does not want to see darker-skinned Tolkien elves or a black Hermione it likely has nothing to do with racism.
I'm not about to start playing mind reading games. The fact that Tolkien wrote the elves as fair skinned and orcs as dark is grounded in pretty racist ideas. This has been shown with more than a few quotes from the books. This isn't something that people have just made up. It's right there in the books.
Which means, if you defend this choice, you are, in fact, defending racist ideas. There's no way around that. You can frame it as "traditionalist" or however you like, but, at the end of the day, there's no escaping the fact that the person is defending racism.
Pretending that simple taste issues (whether I like 4e or not, or Timothy Dalton as Bond) is whitewashing the issue and again, really dismissive. It's great that you don't have this issue. Fantastic for you. Wonderful. But, why does your lack of taking issue mean that I'm somehow wrong for doing so?
The fact that you'd actually defend criticisms of a black Hermione as having "nothing" to do with racism is, frankly, pretty blind. Are you seriously saying that all the criticisms of the play had nothing to do with racism? Not a single person who raised the issue had a single racist bone in their body? That the Twitter storm over casting black actors as elves has absolutely nothing to do with racism? Not a single person complained because of racism?
Like I said, I'm not a mind reader. If someone is defending racist concepts, they are defending racist concepts.
Why they are choosing to do so is not my problem. I really, really don't care. I'm not going to play the whole "Well, I'm not really a racist but" game with people. And, if someone chooses to defend concepts that are pretty clearly racist, they cannot escape criticism. Sorry, but, that's the way it is. Playing the "badwrongfun" card doesn't excuse using racist tropes in your game. I'm sure people have fun with it. Bully for them. Just because they enjoy it doesn't suddenly make it not racist.