I liked the post that took the DM factor out of it to point out players generally want their characters to be balanced “with respect to each other” so each of them gets a chance to have the spotlight. That is a statement I can get behind, and is entirely different than “balanced with respect to encounters (individually or in series).”
Unfortunately, nothing I have ever seen suggests that "spotlight balance" is actually effective in any meaningful sense. It's a lovely idea, that doesn't actually work in the D&D space, because one of the classes has as its thing "bend the rules of the world to do what I want done."
You cannot have meaningful "spotlight balance" when one group of characters gets to decide how concentrated the spotlights are, and the other group is
dependent on them.
I will stick by my premise that people are psychologically poor at probability and if you tell someone they should win 50% of the time, they think the game is rigged against them unless their actual win rate is 70%. Similarly, tell someone they have a 90% hit chance and their brain turns that into “I can’t miss.” So no matter where you think “balanced” is, my contention is that psychologically, players don’t want whatever that number you might choose is, they want the odds titled past that number in their favor for it to “feel” balanced. This isn’t an indictment of players, it is human nature.
Statistical testing has shown that a player success rate of about 60%-65% is seen as "normal", so it is quite easy to actually design a game that fits human psychology. But that's not the primary area that balance matters in. The primary area that balance matters in is whether players feel their contributions to play are of reasonably comparable impact to other players. There's a reason "Angel Summoner and BMX Bandit" comes up in balance discussions all the time. BMX Bandit rightly feels pointless in his "crime-fighting duo" because Angel Summoner's angels literally do everything,
including letting BMX Bandit actually participate.
It is one of the things that I feel makes low level play “better” - hit points are low, dice are swingy, and if I am rolling every roll in front of my players, it is likely one of them drops to 0 hp. Rolling in front of them lets them know I am neither cheating “in their favor” (their victory is earned) nor against them (they are not being screwed because I am angry/a jerk/want to “win” once in a while).
Whereas I emphatically think low-level play makes things dramatically worse, because players can't learn from their mistakes, because the harshness of the consequences ensures that their investment is thrown away repeatedly, and because the swinginess of the dice means that their preparations and choices barely matter--luck will define everything, period. That's not a way to
produce players who want to participate. It
depends on having players who already are gung-ho and willing to forgive quite a lot to get to the good stuff.
I am also an advocate for multiple failure states, morale checks so not every fight is to the death, and alternate dight endings like if one of the PCs goes down, giving them an option to surrender, collect their dying, and leave. Any intelligent enemy would prefer to save his resources and allow the PCs to slink off in defeat rather than have to fight to the death. Of course, the se one time the PCs attack, the BBEG is less likely to be forgiving!
In other words, more reasonable villains and less “tactical combat to the death!”
Sure, it would be nice if that happened more often. Unfortunately, the very same folks who advocate for the "HP are low, dice are swingy" etc. are also, nine times out of ten, the people who advocate for utterly merciless opponents, and for shows of mercy to bite the PCs in the butt (e.g. you let the goblin scouts go...so they immediately run back to camp to warn everyone else and thus the challenge increases tenfold), and for no good deed to go unpunished while evil ones reap rich rewards, and for authority figures that are obstructionist and obtuse in the extreme and generally just an obstacle to overcome. And then such folks complain that their PCs are always murderhobos who take no prisoners, backstab every ally, steal everything that isn't welded down, and go on murder-sprees whenever any authority figure opposes them.
There is no inherent link between the two, I admit, but I've seen this pattern crop up way, way, way too many times. DMs reap what they sow, and they teach their players through their DMing. Far too many DMs don't seem to understand this.
And FWIW, I have found that one PC going down (not necessarily dying) every 4-5 game sessions or so, with death coming only when the party is foolhardy or the dice are particularly uncooperative (perhaps every 5 to 10 times a PC drops the dice might do him in - I haven’t kept track - foolhardy actions are far more frequently the cause… in one case I took the player aside after the session and pointed out several implicit warnings and a couple explicit warnings he had blown past because I wanted to help him recognize the implicit warnings in the future… the explicit ones I reminded him of but didn’t feel I needed to further explain … when I said “this course of action is likely to get your character killed” I wasn’t joking) seems to be about the right amount for players to “feel” the stakes are real without getting frustrated.
Would you believe me if I told you that this is far more representative of the oh-so-maligned 4e than 5e? I've seen
at least three almost entirely separate TPKs in 5e--at the allegedly better low levels you describe. Meanwhile, I've not only had my own character die in a 4e game (thankfully, he got better through the party's efforts), I've also seen three deaths and two
very near misses, all of that in one campaign that sadly didn't get past level 5. (The DM had to stop due to a family crisis that would require basically full-time attention for the indefinite future.)