So . . . why not play 4e? It's encounter balance mechanics sound like it's exactly what you're looking for. That's not meant as a poke or anything. We played 4e for 3+ years and had fun with it. I prefer the quicker (easier) combats of 5e that let's us tell more story in a single session, but if I was looking for edge of the seat combat in every encounter, 4e (or maybe 13th Age) is where I'd go.
(1) Because 5E works just fine for my playstyle.
(2) Because from what I know about 4E (including a grand total of about five hours of playing it), I believe that it's pretty much the exact opposite of what I want. All the stuff I hate most in 5E such as boring predictable encounters that are just pure combat slogs come out of 4E. (There are some things that I kind of like which come out of 3E/4E as well, such as reactions.) I also hate magic item dependency (which thankfully is not a thing in 5E) and gamist class mechanics. 5E is nicely reminiscent of 2nd edition, and while it has problems, so far I've been able to patch all of those problems. I don't think I could patch 4E into something I'd want to play.
Everything I know about 4E tells me that it's a terrible edition in which to attempt Combat As War.
I'd be interested to see an example of the 3x and 4x deadly encounters you set up to challenge your PCs and what the PC builds are. Are you finding this is necessary because you only run a couple of encounters per day? It blows my mind that you are using such encounters on a regular basis.
Sort of. It has a lot to do with my approach to the game, which is fundamentally simulationist rather than gamist. Providing a good experience to the players is a part of the DM's obligations, so I have gamist obligations and I take them seriously enough, and for this reason I don't mind designating PCs as "weirdness magnets" in the metagame... but fundamentally, the things that you need to do in order to force players through 6-8 Medium
discretized encounters per day stick in my craw as exactly the kind of stuff that I loathe and which destroy my suspension of disbelief as a player. "Oooo, suddenly a vampire jumps out of a closet and attacks you! Roll initiative! No, I don't know why that vampire was just waiting in there the whole time you were fighting the ghouls in the next room fifteen feet away." I exaggerate out of disdain.
Probably the single deadliest encounter my players have had, which took place over the course of about two sessions (the last half of one session and the first half or so of the next) is when their spelljamming ship was returning from exploring potential sites for a colony they wanted to establish, and they ran into neogi slavers in a Death Spider. Since they were in fact flying a stolen neogi hammership with the flag from the (neogi-affiliated) Temple of Might, the neogi ship immediately accelerated towards them. There was an elvish Man-o-War in the distance too which also accelerated towards them. I expected this to be a case of flight, maneuver, maybe even combat if they neogi overtook them and attempted to destroy and/or board their ship and the players would have to fight a holding action until the elves caught up and broke up the fight. (The Elvish Imperial Navy plays a role similar to the British navy circa 1900 in keeping the peace and preventing piracy.) Then it might turn into a legal battle of some sort, or the PCs might be forced to pay kresim (10,000 gold ransom) to get the neogi to leave them alone. What I did
not expect was that the PCs would ram and attempt to board the Death Spider. I don't remember exactly what the PC levels were at this point, but it was something like an 11th level Necromancer and twenty or so skeleton archers, a 12th level Barbarian, and an 8th level Death Cleric, and a friendly NPC Grey Slaad (who was there because he thought it would be fun to "slay evil" like a paladin, without having much real conception of what evil really is) who was working with them at the time, plus their crew who were there to fly the ship and work the heavy weapons. Opposing them were 24 umber hulks, another 20 or so neogi, a neogi cleric flying the ship, and an 8th level neogi wizard who was the captain, plus about three times as many heavy weapons as the PCs and human slaves to work the weapons. A quick BOTE says that's approximately 20x Deadly, and despite the fact that the elves were coming and that I roleplay enemies rather than metagame (e.g. umber hulks would emerge naturally onto deck at a random rate, 1d4 per round to represent their varying distances from the exterior exits at the time when ramming commenced and varying decision processes, instead of perfectly and seamlessly coordinating to emerge unrealistically all onto deck in the space of six or twelve seconds, even though it is technically possible by 5E rules to get 24 umber hulks through two 10' wide doors in less than two rounds by perfectly coordinating motion)--despite all that, I fully expected the players to TPK themselves.
I'm reluctant to go into unnecessary detail, because I generally find that there's nothing more boring than hearing about someone else's game, but let's just say that to my surprise and due in part to Evard's Black Tentacles and Spike Growth and good use of ship's geometry and good player tactics such as clearing the heavy weapons first and liberating the slaves, and in part to a wild magic zone which made Evard's Black Tentacles twice as big as usual [i.e. my player misread the description of EBT and I didn't double-check; so when I figured out next session what had happened, instead of nullifying a whole session's combat I just charged them a karma point and retconned it as a fluke wild magic zone which doubled spell radii and would also have affected the neogi wizard's Fireball at them, if not for the fact that the PCs had Counterspelled the Fireball]--because of that
and the failed morale checks from Umber Hulks who were half-dead from withering missile fire and EBT and Spike Growth before they even made it onto the PC ship, the PCs were able to repel the initial assault. Then they actually launched a counteroffensive (!!!) onto the enemy ship, trying to invade the interior of the ship. That didn't go so well for them, and in fact that Grey Slaad "paladin" leading the charge got turned into chunks of flying hamburger as soon as it ran inside, at which point the players smartly pulled back and regrouped. But they still wanted to take over the whole ship, and they rejected the 10,000 gp kresim the neogi offered them to leave them alone (mostly out of ignorance as to what offering gold actually meant--I expected them to stop and try to find out what was going on with this gold, but they didn't--but they took the gold and kept attacking) and here's the part where I went easy on them in some ways:
The logical next reaction for the neogi, if they thought the ship was actually going to be captured, would be to blow the ship. That means that continuing to assault the neogi could have no good outcome: either the PCs get slaughtered just like the Slaad did, or they are on the verge of success when suddenly everything gets snatched away from them by a ship exploding under them. I have mechanisms in place whereby that would not necessarily destroy the campaign per se--both the existence of character trees and my metagame rules for karma could have saved them, but I knew that either way it would have felt unfair, like a DM cheating to "punish" players for doing well against his pet villains, so I wanted to avoid that outcome. Since the pilot of the PC ship was an (ex-PC) NPC under my control, after they got the gold I had him start frantically shouting for the PCs to get back on the ship, and when the neogi and umber hulks renewed their counter-counteroffensive, the PCs got back on... and the pilot took off. At least one of the players was disgruntled when the pilot chose to disengage and fly away from the neogi ship instead of just strafing it with heavy weapons and then ramming again in a new location, because he really wanted to take over the whole neogi ship, but as explained above I knew that could have no good outcome even if they "won"... so instead the encounter ended, and the next encounter was with the elven Man-o-War, where the player inadvertently set in motion the events which eventually ended up getting their planet invaded from outer space and their home kingdom dismantled.
So, most "days" have zero encounters, and some days have one or two or three, but it would be a rare day indeed where you had EIGHT encounters in a single twenty-four hour period. That's, like, something that might happen in the middle of a siegebreak, or a planar invasion from Baator, or something. It's not something you'd expect to see on a single day. Sometimes the encounters are trivial (not even Easy). Sometimes they're not. There was a short span of time in which the players had multiple encounters per day (they were trying to clear their floating island of ropers and phase spiders, while avoiding the rocs, so they could land colonists on it) but there were about 70 of each kind of monster on the island, and I really didn't want to play out 35-70 combats which were slightly different from each other but mostly the same, I instead made the players a deal: "if you choose to do a combat where every roll the enemies make is at advantage, and every roll you yourself make is at disadvantage, we'll call that your unluckiest combat of the day and handwave nine more similar combats, so you'll actually clear ten times as many enemies as you actually fight, and you'll get ten times the XP."
That first encounter, when I computed the deadliness afterward, was Deadly even before accounting for the bizarro advantage rules in place. The one thing I remember clearly is that the ship lost its pilot to phase spider venom, and someone thought fast and barely made an initiative roll to jump onto the helm and replace the pilot before the ship fell out of the sky, and then something happened
that pilot and they nearly died again. The party was
this close to a campaign-ending catastrophe (everyone including the 0th level crew takes 20d6 falling damage and the ship is probably destroyed) but it didn't happen, and instead the players got this awesome story, a ton of XP, and a site for their space colony.
You can tell me all you want that even 1% chance of TPK too much for a combat in a "safe" campaign, but I'd rather stab out my own eyeballs with a pen every week than play a game with dozens of "safe" combats every week. If my players want to play that kind of game, I'd rather invest that time in writing them a 5E CRPG instead of running a CRPG manually. Abnegation is great fun for CRPGs but it's not something that actually needs a live DM.