The Hill Giants don't have to have full surprise, they just have to get lucky on initiative or even just survive long enough to try to splatter the mage (I've seen plenty of inexperienced parties NOT focus fire, a big mistake in this scenario).
But, in general, I 100% agree with you. Hill Giants tend to not live up to their CR rating against higher level groups - they're dumb (relatively) easy to hit meatbags - that hit hard if the party isn't careful.
So IMO, IT WOULD, be great to have a more nuanced CR system that could account for such things.
I will agree that the numbers multiplier doesn't really seem to really work all that well, especially when talking monsters with CR much lower than the PC's level. But it just ... well really depends. If one of the hill giants had a head band of intellect and was a giant amongst hill giants in the tactical realm, things change. Or if the party is just not particularly good at tactics. I've had players that flat out refused to focus fire and spread out damage as much as possible every encounter for example. If you have Tucker's kobolds, even an outmatched enemy can be difficult.
I do think there should be a paragraph or two on adjusting difficulty for your group and how each group is different. But you could have an entire book on encounter building theory and I'm not sure it would really help all that much. It's always going to be a bit hit or miss, the best they can do is give us some general guidelines so we can figure it out based on the group, number of encounters between rests and how the DM runs their combats.