I'm in the 8' tall is an ogre camp. Giants start at 10'-12' - if it ain't ten foot it's not an ordinary adult giant but that's all giants need at a minimum. On the other hand, the sky's the limit. Literally. Why cut down anyone else's fun?
I agree that people ought to be able to paint giants however they want. If they want hill giants to be 25' tall without adjusting the stats, then that is their business.
But the problem for me is that that there is some continuity breaks in the mechanics of you take that to the extreme. D&D traditionally (by that I mean back in 1e) handled size very poorly from a realism stand point. In 3e, there was an attempt to make size matter, which on the whole I'm happy with, but the system does have some realism problems (like most systems) for things below Tiny and larger than Huge. Even (maybe especially) if we assume the fantasy giant with fantasy musculuture, at some point I for one start asking, "How is a 6' human really threatening this creature with its little teeny sword? How does a human expect to survive being stomped on by a creature that weighs 40,000 pounds, much less hit by an axe weighing multiple tons?"
And the 3e scale even ends at around 40'. Unless you start creating rules for transcollosal creatures, you don't have a lot of difference between something that is 40' high and weighs 50,000 pounds, and something 400' high that weighs 5 million pounds. At some point you have to say, "Wait a minute, this creatures skin is 5' thick stone. It's wearing a shoe made of iron that is 2' thick, and the creature is so heavy that that iron plate crumples and moves like stiff cloth, and a 6' tall creature can't even reach it's ankle. How in the world is a sword, even a magical one, even a threat? Better yet, why is the wasp swarm I conjured even doing damage? And it's got a 40' long foot a 200' long stride, and exerts about 6000-10000 pounds of force per square foot just when its standing on you. The game can't even model how this thing moves, much less combat with it."
We just don't have rules for even modelling that in the game. You could create them, and they might make for an interesting game/scenario but they aren't something that the game attempts to model in any fashion. If you want to get into that sort of scale, you are basically on your own.