I'm having trouble seeing how D&D was ever this weird transgressive thing, particularly post AD&D Second Edition. Like in the broader context of our hobby the game has always been the safe, conservative (in terms of design ethos not politics) choice. The game we all traditionally leaned into when we could not decide what we wanted to play. It's our equivalent to watching NCIS or Law and Order.
I don't mean that negatively by the way. Those shows are entertaining. So is D&D. They're just like safe.
In the 1980s, there was a big panic over satanism and D&D, with conservative religious figures saying it was teaching kids to practice the occult. Where I grew up in NYC this never really came up, but I've read people in more conservative areas of the country had D&D banned at school, etc.
Those tanar'ri and baatezu? They were renamed from 'demons' and 'devils' because of this. The assassin class and half-orc race (this is well before anyone dreamed of calling them 'ancestries') were removed.
Rock used to be the music of youthful rebellion, now it's your dad's or granddad's music.
“The Moving Finger writes; and, having writ,
Moves on: nor all thy Piety nor Wit
Shall lure it back to cancel half a Line,
Nor all thy Tears wash out a Word of it.” --FitzGerald's
Rubaiyat of Omar Khayyam
Your point is excellent, though; TSR, then Wizards, and then Hasbro, as a major corporation, pitch their stance to whatever they think will be best for the bottom line. A lot of their stances on political issues (on which I
really don't want to say anymore) probably have more, IMHO, to do with where they think the 'center' is at any given time than any actual convictions on their part.
The reason it takes over the hobby is network effects; until the pandemic you wanted to play in person with a group of people, and if D&D has half of the market and the next one down is Call of Cthulhu with 15%, well, if there's ten roleplayers in your area you've got 5 people for a D&D game, maybe one or two to run Call of Cthulhu, and finding someone else to play, say, Palladium or Ars Magica may take some travelling.
(Most recent figures I could find:
D&D 5e strengthens its market share)