This makes me wonder about the disconnect between the progression math and the player psychology.
Number Go Up is tried and true game design for progression, but what if D&D focused a bit more on horizontal growth instead of vertical growth? That is, you get new abilities that are about the same power level more than you get new abilities that are more powerful. For the fighter, it's like you get more magic weapons, but each weapon is still only adding 1d8 damage. More magic armor, but it's adding resistances or giving new ways to impose disadvantage on the attack rolls. Same power level, just more options for it.
Maybe we still have the Number Go Up, but it's big jumps. You get an entire tier or 5 levels at a time or something that really makes you feel like you're part of a new world. But then maybe you plateau there for a few levels. And you don't get those just from XP accumulation, but from big, momentous events. Like, if these could tie to resolving a PC's personal conflicts you could be very Shonen about it, where having an emotional breakthrough suddenly lets you punch out the villain who was beyond your ability just a few minutes ago. Or you could tie it to treasure for a dungeon crawl vibe (it's not just a +1 sword, it's a +5 sword!). Or you can tie it to slaying monsters for a bit of that old XP-for-killing-monsters vibe, but it'd be like, SLAYING A DRAGON or DESTROYING A LICH, not just bopping your 5th orc on the head.
I wonder if that would allow DMs to give progression when it "felt right" (like when the party was getting bored), rather than linking it tightly to play time. You could speed run a 1-20 game in like 4 encounters.
I guess the central idea I'm exploring there is,
how important is vertical progression, especially over time, and especially in comparison to horizontal progression? Does Number need to Go Up as incrementally or as consistently as it does? Is there better or worse rates? Does it vary between groups? 5e today has a pretty well-assumed pace, and some built-in vertical/horizontal transitions...is it working well? Could it work better or be more flexible or more impactful?
Do you remember, back in the early days of "D&D Next," when they actually talked about this very idea? It didn't end up happening there either. Thing is, this approach is really hard. For a lot of reasons:
1. Horizontal progression (hereafter, h-prog) is dramatically harder than vertical, because v-prog is usually pretty easy to summarize (e.g. 4e's baseline half-level bonus), while h-prog is necessarily far more voluminous.
2. H-prog requires significantly more
meaning infused in, which rarely generalizes well. That is, as you say, "slaying a dragon" or "destroying a lich" are examples that are generically meaningful, but really good h-prog usually requires being able to respond to
unique things, like (to steal from my DW game) "absorbing the power of my fiendish ancestor from a group of exploited tieflings, so they can chart their own destiny without its influence" or "being forced to see the higher-dimensional space my reality is embedded in, and as a result, developing rudimentary magic I don't understand."
3. As noted by I'm A Banana, h-prog starts off at a satisfaction disadvantage. Make Number Go Up is straight-up Skinner Box stuff, it tickles the little grey cells automatically. H-prog has to prove
that it is progression in the first place, otherwise it just feels like "more of the same."
4. H-prog has a problem you rarely see with v-prog:
choices between alternatives. One of the big problems with Battle Master design, for example, is that a lot of maneuvers...well, frankly they suck by comparison. They just aren't worth taking. V-prog doesn't have that same "it isn't worth taking" effect, because (at least in most D&D implementations) you don't get a choice, you just get Numbers Go Up. Because h-prog permits choice, it requires that the choices be fairly close to equal utility, otherwise you run into serious diminishing returns, which can make "higher" (broader?) levels deeply unsatisfying.
There are probably more but I'm tired and need to lay down. Point being, horizontal progression has some pretty serious design hurdles that just aren't present with (most implementations of) vertical growth.
the 5e skill system designs against both of those options in service of bounded accuracy.
Personally, I prefer skills which can grow a little if you deeply invest in them, but which are already innately quite broad--so picking up a new skill is actually quite strong, especially if you can do so while gaining other benefits alongside. "Arcana" isn't just a list of narrowly-defined specific applications. It's the one-stop shopping center for anything to do with esoteric magical weirdness that isn't from the gods or from nature. Medicine isn't just for a couple specific things, it's literally anything to do with living sapient bodies (non-sapient bodies would be Nature or, if it's separate, Beast-lore.) History isn't just a handful of dry facts, but genuinely comprehensive humanities, including stuff like law, (academic) politics, military theory, etc.
Under that regime, every skill is a powerful arrow in your quiver, but you don't have to run a Red Queen's race just to get access to their useful benefits.
Having weapons with mechanical hooks to provide room for playing with crit range/crit mod damage dice/meaningful damage type/etc was another example of how horizontal growth requires depth that 5e simplified away.
Believe it or not, I actually have like 95% of a "design your own weapons" subsystem written up. It's based on the 4e weapon groups and properties, but there's not really any reason it
couldn't work in 5e (and, indeed, some of my ideas were used in BG3! I've even "stolen" one in return, which BG3 calls "Tenacity"--the one that deals flat damage even on a miss.) I also went through and evaluated and re-balanced the existing weapons of 4e to fit the "build-a-weapon" format--ironically, developing the format wasn't much harder than adjusting the handful of outlier cases and coming up with new cool properties people could choose from.