Thought exercise
The other day I had heard something from Joe the Lawyer about how spells like knock, wizard eye, etc were important in TSR era D&D because they allowed you to get to that treasure, and it really resonated with me in the debate between OSR (TSR era) and "modern" (WotC) D&D styles.
XP for Treasure
Many of you who have never played TSR era D&D have still heard how you were awarded XP for treasure. This was true. You might get 100xp for the monster, but 500-1000 xp for the value of its treasure. This changed with WotC (well, started to in 2e, but was still an optional rule). People tend to act in ways that foster the reward. Basic human psychology. We do things at work that we are rated/rewarded on. Same for games.
Result
What I saw happening as result was that combat was not only done more often, but it was expected. I saw a shift from "not every monster should be beatable" to "every encounter should be beatable" in attitudes as modern DnD took hold. There was an increased expectation from players that every combat encounter should be winnable because that's how you got your XP. Contrasted with TSR DnD, where most of us had an expectation of play that you avoided combat whenever possible (because it was so dangerous) and found ways to bypass the combat and go to the treasure (where the XP was).
With this shift was the de-prioritization of the Exploration pillar in at least two very prominent ways:
I think there's definitely something this, but I think the reason xp for treasure went away was more to do with the Hickman Revolution, or rather the sector of players who came from sci-fi/fantasy fandom as opposed to from wargaming (which predated the Hickman Revolution) becoming thoroughly dominant, and TSR catering to them.
Even in the 70s, the play culture for D&D was split between (broadly, and in simplified form) the wargamers who focused on it as a game, and the sci-fi/fantasy fans who focused on it more as a vehicle for characterization and telling heroic stories.
For the latter group, Lord of the Rings, Shannara, and other epic quest narratives were bigger influences on their play than Fafhrd and the Mouser, or Conan. And D&D mechanically, certainly by the time AD&D came out, had a lot in it to reinforce the epic heroism aspect. Classes and races drew primarily on Tolkien (secondarily on other sources). The descriptions of Alignment, Hit Points, spell acquisition, Saving Throws, and so forth talked about cosmic forces and the struggle of divine powers, who reinforce and directly protect heroic characters. Once you're past the first couple of levels you're assumed to be a part of (even if unwittingly) cosmic struggles between mighty powers.
If you're trying to run your game as centered on heroism, characterization, and epic quests, xp for gold seems to sabotage the themes you're going for. It's possible to kludge it and look past it, but fundamentally there's a major thematic dissonance.
By the time 2nd ed came out TSR had had major successes in publishing their own heroic fantasy fiction, and as I recall very little of it focused on the protagonists getting enough gold to level up.
But fighting bad guys and smiting monsters; why, that's very much central in heroic fantasy novels, movies, and TV shows.
In 2nd ed we see a very wishy-washy set of rules and suggestions for xp awards. Gold for xp is made optional. XP for defeating monsters remains core, though not much more generous than it was in 1E. Which means trying to advance just through combat is slow and dangerous as hell. The DM is instructed to give quest/goal awards, but not given a ton of guidance on crafting those. Though as I recall, what instructions there are tell you not to give more, or much more, quest xp than there is monster xp. Which is still a recipe for high lethality and slow advancement.
I'm not sure if the design team (much as I respect David "Zeb" Cook) quite realized the knock-on effects of making xp for gold non-core. As you and many OSR writers have observed, xp for gold gives you an objective success measure and goal to incentivize tricking, avoiding, and NOT fighting monsters you don't have to. But this whole orientation, to a person trying to emulate heroic fantasy fiction, doesn't support the kind of fantasy they want. And TSR was trying to support all those players who wanted to play the hero of their own fantasy novel.
By the time 3rd ed rolls around we can see that they're still doing both monster and quest/goal-derived XP, but they've brought a bunch of math to bear on calculating monster xp and figuring out how to make leveling from fighting bad guys not take so damn long.
Spells
In TSR era D&D, utility spells were important, and often more important than combat spells. Most old school players knows how only a newbie Magic User takes Magic Missile at first level, the real powerful spell to take was either sleep or charm person. Spells like levitate, knock, teleport, invisibility, and dispel magic were very important. Sure, you also had combat spells, but crowd control was more important than DPR: hold person, sleep, stinking cloud, etc. If I were to make a guess, I'd say over 50% of your memorized spells were utility spells. Again, bypass monsters and traps (which there were a lot of), and get to the treasure.
In modern D&D, I'd say close to 75% of cast spells are combat encounter orientated. That's the style of play. Along with a philosophy of "every character should be able to overcome any challenge" (as opposed to how TSR emphasized a team niche aspect), there isn't as much of a need to spend your spells on exploration or utility spells--some other class has a power to help with that.
Ah, let's be real, Sleep is a combat spell. And so is Charm Person, a lot of the time. Often a way to forestall or avoid a combat and recruit an informant or even a henchman. (Charming bad guys and monsters was explicitly one of the main ways to get a permanent henchman in OD&D, though subsequent iterations of the rules progressively weakened that once insanely-strong spell). Smart players know not to take Magic Missile at low levels because it's a combat spell, but because it's such an ineffective one next to Sleep, Charm, or (once the blinding use is codified) Light.
I do think you're right, though, that modern D&D focuses more on making sure every character is good in a fight, i part because combat has come to take up so much of a game session, and they don't want anyone sitting around during fights feeling weak or useless.
The thief
I've seen, many times, how modern D&D players look at the AD&D thief and think it's a woefully underpowered class. From a modern lens, I can see how that might be, because it's being viewed through the "combat all the time" lens. If the AD&D was forced to fight in every encounter, they wouldn't last long. Their specialty was getting out of combat (finding secret doors, sneaking by, etc) and getting that treasure and XP (PP, OL, F/RT). In AD&D, the thief more than held their own in gameplay based on the style as explained above. So when modern D&D came about, with the emphasis on XP for monsters, the design team had to beef the thief up significantly (like applying sneak attack/backstab damage once a round).
Yeah, no, Thieves genuinely sucked. They require generous adjudication in ways the OSR has innovated in recent years to be any good, and even then a bump from d4 to d6 HD is usually necessary unless they're in large enough parties that they just never get into melee. Gary's instructions on how to adjudicate Thieves' abilities in 1E were harsh and taught harshness to generations of DMs. Once 2E rolled around and you could allocate points you could be good at 2-3 skills at low levels, but man, Thieves were weak overall.
One of my biggest pet peeves with "modern" DnD design (4E and later) was the move away from spells as general utility. Yeah both editions do have utility spells, but nowhere near the emphasis as earlier editions.
TBF, 4E moved almost all the non-combat magic to the Rituals system. Which was in some ways an excellent innovation, as it meant that you weren't forced to choose between holding your own in combat and doing all the useful utility stuff.
