Here is my frustration, they completely slipped past 2 weeks of planning for a big showdown with this NPC. When I told them I would have him autofail the save to avoid the boulder and I gave them 15d10 dmg for coming up with such a good trap. They lost their minds saying he should be dead. I tried to reason and asked how they would feel if I sprung a similar trap on them, only to be told I would be pety and doing so would ruin the fun of the game. For what it matters the party is made up of of 5 8th level characters. . A cleric, fighter, Druid, rogue, and a bard.
Im not sure how to proceed, I don’t want my game to stop and implode because they feel I’m being unfair. Should I have just had the bad guy die on the spot?
Improvising damage according to the DMG:
10d10 Crushed by compacting walls, hit by whirling steel blades, wading through a stream of lava.
18d10 Being submerged in lava, being hit by a crashing flying fortress.
Sounds pretty much where it should be to me. Maybe you could have done 10d10 bludgeoning and rolled separately for a bombs (
fireball might work), but that would average out to precisely half a point more damage than just doing 15d10. So let me revise my statement--the damage is
spot on exactly where it should be. Auto-failing a save is being generous.
The reason we have hit points instead of auto-kills is that the character has a chance to take some sort of cover, avoiding the worst of the damage.
Now, as far as giving the opponent 120 hp, if that isn't something that makes sense and gives your players a feeling that if they were at a similar level of skill (maybe 12th-14th level fighter) as the bad guy
they would have about 120 hp, well then everything I say goes out the window because you aren't treating the PC side of the equation and the NPC side of the equation equally, so you're going to have problems until your group makes up some sort of rules for exactly how this stuff is going to be done. In this case, you might need those rules to be explicit and written down.
But it will probably make it go a lot easier for you if you just make PCs and NPCs follow approximately* the same rules, so the players can know what to expect, know that they and equivalently skilled NPCs can survive the same sorts of scenarios, they and NPCs can pull the same basic sorts of stunts, etc. It doesn't sound like they want you to tell them a story. They want you to present a playground for them to explore and do what they want to in it. You'll have to decide if you are interested in doing that, or if they are interested in switching to story mode. Or, you can make different parts of the game do different things. For instance, it could be that during defined stories (which you would tell them are defined stories) you make stuff up for the story and it doesn't have to be consistent or make sense, and during the exploration mode (again, defined) the world is designed to make sense and have PC/NPC equivalence. But that's difficult to do in one campaign.
* That doesn't mean you have to build all the NPCs with PC classes, but if there is no way for PCs to get roughly equivalent abilities to NPCs (and vice versa) any sort of world verisimilitude flies out the window, which isn't what it sounds like your players are wanting.