Is it broken or is it a case of people refusing to accept the baseline assumptions of cr and then proclaiming that it doesn’t work?
It's broken. It's impossible to rate 4 PCs of the same level, because 1) classes vary in power, 2) subclasses vary in power, 3) players vary in skill. You can't assign a number to a monster and have it fit all the different combinations of class/subclass/player skill that are out there
It's easy to have expectations of CR that the CR presented in the game can't possibly meet. True "Difficulty" or "Challenge" is such a squirrelly target in a TTRPG.
The mechanic of CR in 5e really just compares 2 things: HP vs. Damage. For each side.
That's all CR is really made to do: this monster won't one-shot or two-shot a given PC, and a given PC party won't one or two shot this monster, ending the encounter before anything interesting happens. It's a mechanic that paces a fight more than anything else, and that's just a measure of HP lost for each side over rounds. That translates into a "challenge" in that it creates a kind of timer for the fight, where if the monster isn't taken out quickly, its damage begins to have a chance of knocking a PC down to 0 hp and starting a death spiral for the party. So there is a bit of a "skill floor," below which the tension rises.
Luck can sway that. Clever play can sway that. Battlefield conditions can sway that. Binary abilities like
hold person can sway that. Expectations sway that, too - some folks are like "If this monster can't get a TPK on a party of this level, is it REALLY a challenge of that level?"
But, CR has never been an exhaustive measure of every factor that could go into challenging a party. It's not trying to be. It's not "broke" in that way.
None of that influences Challenge Rating because those are all valid strategic plays that deeply depend on the context of the fight. If you are fighting on a tall bridge and you can knock a monster off of the edge in 1 round, that's fine. That's good play. That makes the fight easier, but the CR is a method for rewarding XP, so it's almost like getting a little XP bonus for being smart and using your environment, which is great. An optimal party is an "efficient" party, with a good XP to action ratio. And the CR formulas delivered a situation where good play like that was rewarded - you just vastly over-performed your damage expectation for the round.
It's understandable that people expect more from the mechanic, because it's called CR after all, but it boils down to something very simple: HP vs. Damage.
So when I tune encounters, I follow Xanathar's recommendation for a "hard" encounter (I assume +50% PCs, so this typically means more monsters). Hard in practice means that the monsters start out a fight with a pretty notable advantage in terms of the damage they deal (more critters, more actions, more dpr), and the party's damage output doesn't outpace monster HP for another round or two. This means that my party has really started to value defensive abilities - if they can weather the initial onslaught, either through proactive protection, or through healing, they are still likely to be victorious. But, they can't ignore their defenses much. If they do, they usually enter a situation where someone's down to 0 hp and they've still got a critter or two out there that they have to deal with.
That's not "hard" in the sense that it is difficult to overcome. It's just a little more hostile, a little more risky, demanding a little bit more of the players to meet that skill floor. It's not particularly difficult for a player to do that, but the do need to pay attention a bit more than "average."
This design naturally punishes more "reckless" characters and more "reckless" play. A flat-out damage monster of a barbarian would actually be a pretty weak character in my game, since a lot of their damage would be overkill (most encounters have multiple monsters), and they'd drop pretty quick if the party didn't try and prop them up.
I'm OK with all this - it is actually very on brand for the vibe I'm going for. A game inspired by a Fromsoft title should absolutely reward cautious play and value defensive abilities and punish those who just try and hammer the attack button as hard as they can.
But, if a player really liked big damage numbers, they might feel like my encounters are too hard ("I always drop in round 1 or 2, that can't be the intent!"). Measured in PC's dropped to 0 hp, my encounters are fairly "deadly" (roughly a 1 dropped character per 2 fights ratio), but measured in TPKs or actual PC deaths, my encounters are a cakewalk (doesn't happen, not MUCH of a real risk, though there have been a tough moment or two).
Because my group's excited for 2024, I'm letting them weave in the new rules, but this also means that the monsters I'm making aren't going to be hitting as hard as they should. I ride that line of "
just enough to make them squirm." They won't be squirming as much now, making my encounters easier on average, and really damaging the vibe I'm going for.
But, like, WotC never did a great job of clearly defining the boundaries and expectations of CR. And, to be fair, understanding those bits is VERY designer-wanky. Casual DMs probably don't need to apply. Even experienced DMs like the folks here are surprised to learn that CR doesn't actually care about 6-8 encounters per day or that CR doesn't need to worry about size or energy type or specific subclass power levels.
And add to this that the actual math of the damage to hp ratio was off at higher CR's (monsters at higher CR's did not hit as hard as they "should" - I gather this mostly from interviews, since most of my campaign is pretty low-level)....
CR is broken, just not in the way that most people assume it is broken.
