I’m not talking rules or mechanics. I mean in terms of theme, tone, and aesthetic.
I don't think there's any real way to separate these things. The art, theme, tone, and aesthetics
imply what the setting
is like. The mechanics
dictate how the setting
actually is. You can have all the story snippets and art suggest heroic or epic fantasy all day long, but if the mechanics tell you that you start with 1 hit point and even a cat will deal at least 1 hp worth of damage...you're not going to get anywhere near emulating that art. Comparing a 1st-level character from older editions to a 1st-level character in 5E will tell you a lot about the shift in the tone and theme. Comparing healing rates of AD&D to 5E tells you infinitely more about the implied world than the shift in art.
To you, what are the main ways D&D has evolved it’s t and aesthetics since you first started playing?
I started in '84 with B/X, but quickly moved to the "more adult" AD&D. So that's my frame of reference. Color covers and black & white art that showed characters engaged in dangerous activities surrounded by typically small- or human-sized opponents, or squaring off against something huge and menacing. And the characters showed fear. They were typically afraid of what was happening around them. Cowering, gasping, shrinking away.
The cover of the Moldvay Basic Set has two characters against a dragon-thing. The woman casting looks horrified. The dwarf standing next to her is leaning away with his shield up guarding his face, spear ready to thrust. The AD&D DMG's guide has an efreeti squaring off against three characters. The unfortunate damsel in distress trope aside, it's three PCs against a huge, tough monster...and all the PCs are shrinking away. Dungeoneer's Survivial Guide has a lone climber being menaced by a few tiny creatures. MM2 has one character against a giant or ogre. The Wilderness Survival Guide has three against some kind of giant or ogre and they're clearly threatened by it. One's already captured and about to be dropped into a ravine.
Compare that to the 5E covers. The PHB has two against a giant. The PCs are on the attack, one mid leap, they show nothing like fear or trepidation. Though the MM cover is more like the older stuff. Two characters against a beholder and it looks like they're running away.
The interior art suggests much the same. In older editions, B/X and AD&D, there was more threat, menace, death, and dying. In recent editions, especially 4E and 5E, there's more action heroes fighting fantasy monsters.
It’s hard to pinpoint how, but I feel that the implied ‘setting’ or ‘genre’ constantly evolves. I’m not an OSR style gamer generally, and my memories of play way back then are mixed in with being a kid, so everything was different just because I had a different lens. It swiftly feels more cartoony or modern American Ren Fair to me than it used to rather than anything medieval (which is fine — it is it’s own genre, not a documentary).
I think it's fairly easy to spot, myself. There's a distinct shift away from being rooted in the pulp fantasy, swords & sorcery, and weird fiction of the 1920s through 1970s. Appendix N. Less Conan and Fafhrd and the Grey Mouser. A shift from humanocentric to the "Mos Eisley Cantina" effect. I'm glad for the increasing diversity of human representation in the game, that's wonderful to see. I do think there's something lost when you take every vaguely human-like creature and make them humans with funny hats or facial prosthetics. You're taking something magical and making it mundane. Don't show me the humanity of the monsters, show me the monstrous in humanity. To me, that's infinitely more interesting.
I think that's a large part of the tonal shift. Making the magical mundane. That shift away from even a bad attempt at rooting the game in the medieval to the cartoony Ren Faire you mentioned. Things becoming more abstract and less rooted in our understanding of history. As you say, it's a game not a documentary. But it feels more real, more immersive, when it hews closer to history than self-reference. For something to be special it needs to be rare or unique. Dragons aren't rare, magical, or special when you can fight one every level or play as one. Magic isn't rare, magical, or special when 9/13 of the classes use magic...and there are magic wielding subclasses for the other four. Magic items aren't rare or special when you end up with so many you need to sell them off to make space in your bag of holding. Skipping over things like how much you can carry or worrying about food and water or light sources is also a big shift. Sure, a fair amount of people skipped that back in the day, but a lot of us didn't.
For me, Dungeon Crawl Classics nails it. Rooted as much as you can in the medieval, make magic rare, scary and dangerous...and suddenly it all just pops more. You can go gonzo and weird with it
because it also tries to be more rooted. It's only special and magical by comparison. If everyone and everything is magic, nothing feels magical. To quote Syndrome, "When everyone's super...no one will be."
Anyway. A lot. Enough rambling.