What are we quantifying with CR? The "threat level" of a given monster? But that's highly contextual, so are we suggesting that we imagine some perfect scenario for the monster that may never happen in play? Or are we somehow accounting for the context? Do we define threat as a % of resources spent (including hp)? Do we define "deadly" as 100% resources spent, or is 100% resources spent a "normal" encounter, because we don't roll initiative unless it's frickin'
worth it? Do we account for player skill? Environment or battlefield concerns? What a monster "should" be ("there's no way a lich is CR 7, they're just
too badass")? How do multiple monsters figure in? Or do we flatten context a la 4e so that the math works better and everyone's playing in the same room with the same guards on (a la "combat as game" rather than "combat as war")?
And then how do we use it? To build encounters in a vacuum? To build an "adventuring day" of several encounters? To construct a dungeon of combats? How do we weight monsters that might challenge you via conditions? Do traps factor in? Is this just a system for rewarding XP? If I have a difficult skill challenge that uses resources, should that get a CR?
CR cannot be all these things. Some are just contradictory! And any exclusion is just going to leave someone somewhere saying that CR is broken and doesn't work, because it doesn't work how they imagine CR should work. Even if we can rigorously define and mathematically perfect CR for a certain subset of these answers, it's not going to be used that way in practice by players who want it to be something totally different.
The most useful definition of CR to me as a dude who is more narrative focused is probably "Throw this at a party of this level and you probably won't get a TPK." Sort of like a "you must be this tall to ride" indicator. But even that misses what should be "boss-level" encounters that maybe
should risk a TPK, just not like in Round 1, maybe more like Round 5, and more trap-like monsters that aren't really meant to be fought as much as just either dealt with successfully or not.
@mearls's system defines a Challenge Point as a % of resource loss for a single character, and I'm already kind of in the weeds, because that doesn't tell me what my players will be
feeling when we fight the orcs. I want them to feel like the orc horde is an overwhelming force of nature that no army can stand against as it crashes against the small, weak walls of the outpost, that they can perhaps dispatch this pocket of orcs, but even thousands of trained soldiers could not end the threat. And maybe level informs the scale of the pocket they can take out, or the waves they can cut through, or the kinds of orcs you could fight (and I think 4e had the edge on this given that its math chassis could wear many skins).
That's maybe a tall order from 5e, but when I think about the "threat level" of orcs, that's what I think about. The threat level of an orc is "You never want to meet a lot of them on an open battlefield." The threat level of a kobold is "Don't go into their lair if you value your toes." The threat level of an ancient white dragon is "you cannot defeat the concept of an ever-winter with a sword or a fireball." The threat level of an illithid is "you're going to watch helplessly as it eats the brains of a party member or two and runs away" (or maybe "cosmic ancient evil that pops like a balloon."). In terms of resource loss for a single character, you're talking about maybe "10% times the number of orcs", "Toes are about 10%, but the more your pursue the worse it's gonna get," "infinity, but there's probably a mcguffin that turns it into 100% of your entire party of 3-7 PC's unless they use good strategies", and "100% for Lidda and Miallee, but only traumatic memories for Warduke and Rath."
The numbers today only tell me when I get to run that kind of story without killing my party and experiencing DM's Remorse instead of Fun Times. And, I guess on the other side, when I can't run that kind of story anymore without it being a cakewalk. Probably. Ish. Under certain circumstances. It's fuzzy.
IDK, maybe the idea that we run an orc in a vacuum where it can be quantified in Danger Numbers is an idea whose time has passed? I, for one, don't need more granular math. For 5e specifically, I
could use some guidance on the numbers I need to use and how to use them. If I want my orcs to have that vibe, how do I use the orc stat block to achieve that? And what risks making it too easy or too hard? How do the mechanics make that feeling? And about what level should the PC's be? And how big should the pocket be vs. the orc horde itself? And what about the orc general, is that a solo or no, I guess it would be surrounded by troops, right? for the vibe? Or, even a good example of them (here's Orc Horde and Pie Keep!). But "Challenge 1/2" don't really give that to me.