Some monsters are stronger against more PCs and some are significantly weaker the more PCs there are.
There is no way to account for the changing effectiveness of monster abilities except by having a CR attuned to party size. This would be a massive and wasteful undertaking.
You want to play with a party outside the scope of the default design of the game? You make adjustments based on whether you think monsters will be particularly weak or strong vs more characters on your own.
Not just party size, but makeup and encounter specifics. In the OP's case the party got off multiple fireballs against a low dex monster, presumably getting more than one giant (my impression is all 3) in the blast each time. Add in some lucky rolls on the side of the PCs, some bad rolls on team monster and **** happens. If this encounter had been against 3 different CR 5 monsters such as Fire Elementals, the encounter may have ended completely differently.
Nothing is perfect, I've run multiple groups now and what one group would find a challenge, another group wouldn't even break a sweat. Other times encounters I've calculated a medium encounter that practically ended up being a TPK. Some encounters I expect to be tough are a cakewalk.
It's the same argument that comes up every time. Do you roll for stats and have numbers significantly above the expected average? Do your 5th level PCs have multiple magic items each including rare magical items? Do the monsters use effective tactics or just approach on an empty field in fireball formation? What's your XP budget between long rests?
It would take something a lot more sophisticated than a simple calculation to take all the parameters into consideration. I think the guidelines we have are okay-ish other than the fact that I ignore the numbers multiplier but you can't ever boil this kind of thing down to a science. If it worked better in previous editions, it was because the expected party build was more constrained and predictable. They flat out told you in 4E what plus to weapons you were expected to have based on PC level, that progression was built into the math. With 5E? Things are all over the place between high ability score generation methods, feats, magic items, just general DM acumen.
TLDR: Every group is different. I have a multiplier I use for each group, some I use the base calculation, others I increase the budget by 50% or more while also setting up environments to play to the favor of team monster. It's never going to be science, encounter building will always be an art.
