Just going to do selective quoting to avoid a big wall of text
I think then, as now, D&D simply tried to emulate contemporary fantasy tropes. That it happened to be Medieval European Fantasy in the days of OD&D and BD&D was simply due to the fact that other types of fantasy were not as widely-known or as popular then as they are now. By the game reached the AD&D stage, some trickles of non-European influences coudl be seen from the fact that the monk was a core class.
In short, if the tone of the game has changed since it was first published, it is because contemporary notions of fantasy have changed. Harry Potter has replaced Merlin. The movie version of LOTR is better known than the books. Li Mu Bai has ousted Kwai Chang Caine. Cloud has eclipsed Conan.
So, to go back to the original question, I want a D&D that is able to incorporate any crazy ideas that a kid in his early teens might want to add to his game, whether it's a Jedi, Spiderman, a Tauren, an Wuxia swordsman, or a historical knight. I want rules that are robust and flexible enough that anybody could use their creativity and imagination to make a game that they want to play. Sometimes this means adding stuff, in the same why that I added transforming robots and shapechanging knights to my D&D game when I was young (I was a big fan of the Transformers and the Visionaries). Sometimes this means taking stuff away, e.g. someone who wants to run a more "realistic" low-magic game ought to be able to get what he wants by removing some of the more "wahoo" choices.