I think there's an element missing from Celebrim's analysis and that's the presumed campaign length that has become fairly codified into D&D design...
...Is this catering to a certain kind of gamer, or is it recognizing the realities of the player base and designing to that reality? 1e didn't really have a presumed campaign length and, I think, for most gamers, a large chunk of the rules when unused. How many eighth level spells did you actually cast in 1e after all?
Once you make those two decisions - that a campaign will last about X number of sessions, and within those sessions, the players should have the opportunity to experience the entire game - you get your rate of reward.
That's an interesting set of thoughts. The longest campaign I was in went 4 or 5 years, and probably 120 sessions over that period (total play around 600 hours). It went on for a bit after I left it, but when I left the players were 12th-13th level (or equivalent multiclassing). Now that you mention it, I've never cast an 8th level spell in 1e, and maybe not even as the DM running an NPC. I do remember casting 'Reverse Gravity' as a player, and few other similar spells, but those are I think 7th level.
When I started gaming, the default assumption was that all play was inherently open ended. You might arbitrarily pick a place to stop, but there was no end. The 1e AD&D character progression tables implied they didn't stop anywhere, and that the same rules set would extend out infinitely. The 1e AD&D M-U table ended at something like 26th level or some such, but like every other table it carried instructions for going on. There was no assumption you'd reach 'the end'. In a way, D&D seems to me almost to have been the 'new games' version of the war game - open ended, cooperative, inclusive, and governed by a certain spirit of play (most manifestedly by the inclusion of 'rule zero', giving the referee the right to change the rules in the middle of play).
As a player, it never occured to me that I was missing out on something by not having a 18th level M-U who could cast 'wish' and 'meteor swarm'. Name level PC's were fantastically powerful people capable of overcoming many challenges on there own, and in cooperation could dominate just about anything in the game.
I'd be interested in knowing where the shift in expectations came from, or whether I was just unusual in not feeling cheated by not getting to be umpteenth level. If there was a shift in expectations, the immediate - but perhaps incorrect - assumption I want to leap to is that this comes back into D&D from computer games, which, by necessity must be closed ended and which get nothing from offering content which is not directly experienced.