OK. But what you're describing is pretty much a human universal in storytelling. D&D was not developed by 21st century sociologists. It was a tactical wargame influenced by European folklore and mid-20th century fiction pulp fiction. But even if its cultural genesis was Japanese epics, the Thousand and One Nights, or Indian folklore, it would feature that same trait of making monsters monstrous rather than diverse and nuanced populations worthy of empathy and tolerance.
In other words, the things that are bad about D&D's assumptions are bad about virtually all traditional storytelling. And it should be regarded in that context, and not as some peculiar defect of D&D or its cultural influences.
And if we're examining culture and storytelling at that level, them we should revisit the notion of villainy altogether. Personifying our fears and hates in the character of a villain is a deeply-rooted human impulse. Put a face to our fears in the form of a character, and then vanquish that character. Kill them. It's as old as stories. It also runs contrary to enlightened modern notions of where the source of our problems lie and the best way to deal with them. Should we expunge villainy itself from fiction as 'problematic?'
It may well be harmful. But you realize it's found in virtually all storytelling in every culture on earth, right? When the protagonists discover a new group of people, they demonstrate pretty narrow traits because that's how you dramatize. The new group are brawny or weak, gentle or aggressive, trustworthy or duplicitous, greedy or openhanded. That's how fiction works - you can't give all groups all human traits in equal proportions, or they're no longer distinguishable anymore in a dramatic sense. And if they're not distinguishable, they don't serve their essential dramatic purpose, which is to compare and contrast behaviours.
By expunging this approach from D&D, WotC will be in the vanguard of a new way of approaching popular storytelling, not lagging behind. Something as deeply ingrained in the storytelling of every human culture will take generations to change.
Yes. Systemic change requires changing the system.
Two things to note:
1) the way race is used in D&D does not reflect a "historic" definition of race. Instead it's very much tied to the American use of race, which was created in order to define who could or couldn't legally be enslaved.
Cultures have many ways of dividing and segregating people (class, gender, wealth, religion, etc). But the way race is used in D&D, with its half-orcs and half-elves, ability bonuses and alignments and class preferences, strongly reflects and amplifies the use of race, genetics, and the "one drop rule" in American history.
So when we look at the "traditions" that D&D pulls from, let's be honest and unflinching in that look.
2) D&D is not a story, it is a system used to play games and tell stories. Because it is a system, WotC should be mindful of the tropes and stereotypes they want to see played out, and those they don't.
Let's use the Brothers Grimm for a moment. Let's say instead of Tolkien and Vance, Gygax had been inspired by classic German fairy tales for D&D.
As much as I love the Brothers Grimm, they had very strong opinions about the roles of women in society, and they were anti-Semitic. In publishing these traditional stories, they often changed the roles of women to be more passive (rescued by a woodsman or a magical animal), and actually changed villains to be Jewish. (There's a great Imaginary Worlds podcast about it this week.)
So if D&D was based on Grimm's Fairy Tales, should we have rules that make all Jewish people villains by default, and limit the roles of women?
Of course not! We would rightly take those traditions and boot them out the door! And we would keep all the lovely traditions of the fairy tales (imagination, strange rules, magical creatures) and carry them forward.
We should do the same with D&D.
But most games, stories, superheroes have good guys and bad guys. We can be somewhat sympathetic and understand what Thanos did and why, but he was still the bad guy. The reason The Joker exists is because he's easily identifiable as the bad guy.
If you have a game that (at least for many people) is about heroes saving the day you need a nemesis, an enemy. That's why you have things like a Monster Manual not an Alternative Species book.
The thing is there's only so many ways of having a group that represents evil.
Organization: so you have group. Whether that's Hydra, The Empire or Evil Inc, the assumption pretty much that everyone that belongs to that group is evil. Except what about the child soldiers? What about the people who were just desperate and saw no other choice? How is it different from warring political parties? How many people on the Death Star were just janitors?
Religion: Those darn cultists are at it again. Won't those crazies ever learn? Don't they know that the religion I grew up with is the one true path? Oh ... wait. How is that better? How is some followers of some mythical deity not just a representation of [pick your "militant" real world religion].
Culture: Well obviously orcs are only evil because that's the way they were raised. If we only taught those ignorant savages the correct way of speaking, dressing, and proper manners they could be good citizens. Just like the U.S. shipped all those American Indians off to boarding school to "civilize" them. Ugh. No.
Race/Species: It's okay to have bad guys, they just have to be demons, outsiders, undead, xenomorphs that don't look much like humans. Because everybody knows what you look like defines who you are, right? Somehow it's okay to be evil if you stray far enough from looking like a person.
Personally? I don't have a problem with orcs being evil. They are not humans, much less real. They are the equivalent of genetically engineered species designed to be a destructive force by an evil entity Gruumsh. Is that an over-simplification? Absolutely. So is AC, HP, levels, classes, ability scores, healing ... I'm sure we could go on. If every single ogre has the same ability scores and attributes I don't see why alignment is any different.
Except that's not the end of the story. I don't use "monstrous humanoids" in my campaign all that often so I've never expanded them that much*. But the MM tells you that these are just the defaults (even if it could be more explicit). The DMG talks about how to modify and tweak monsters and NPCs to fit specific stories.
IMHO having the simplified option of good and evil is part of the game. It's part of most "heroic" fiction and games. I just don't think there is one way to have that which is "acceptable" while others are not. Reinforce that the culture presented is the default whether that's for orcs, gnomes, dwarves or drow but that the possibilities are as varied as the campaigns they appear in. But don't get rid of the concept of good and evil altogether because sometimes I just want to be a good guy fighting the bad guys without risking PTSD.
The real world is messy. I want to be able to play a game to escape reality for a few hours now and then.
*Most bad guys in my campaign are human, although the real power behind the scenes is sometimes not human.
Oofta I think you are overcomplicating things.
All a villain needs is motivation. As long as a villain has a realistic motivation, it doesn't matter what race, religion, or culture they are.
If an adventure expects you to kill an orc just because it's an orc, that's racist.
If an adventure expects you to kill an orc because it's been raiding local farms, because it's sacrificing people to Gruumsh, because it wants to prevent the characters from achieving their goals... That's motivation.
How do your players know it's okay to kill the humans you put in their path?
