Let's take your Elephant example.
An Elephant can kill a human in seconds. For those who have not seen elephants up close, they are ferocious in combat. I've seen an elephant fed fruit in a bag and it was extremely vicious for those few seconds it takes for it to rip the bag open. It almost literally scared the crap out of some of the people who were nearby watching and caused them to jump back, it was that vicious in such a routine activity for the elephant.
A real life human in armor wouldn't stand a chance. There's a reason that Elephants were used as war mounts in India for millenium.
I feel that the large monsters that can pick up houses in D&D should be that way too.
For example, the Ogre (or whatever it was) in LotR. Slow moving, but when he hit a creature, even accidently, it went flying across a room and smashing into a wall.
That's what larger creatures should be, especially the non-slow moving ones.
Dragons that weigh 30,000 pounds should squash even an epic level PC to the ground under their claw for massive amounts of damage. But, the Epic level PC is still epic. That PC should while hurt, like Hercules be able to lift the massive claw off of him, or like the Gray Mouser, be able to squirm out from under, or like Gandalf fighting the Balrog, be able to use his magic to form a magical force shield. Still pinned to the ground by a dragon and able to use magic, but not killed.
The problem with 4E is that it has no flavor.
There's no GARGANTUAN in Gargantuan sized creatures.
There's no MAGIC in spell casters (everyone has similar level super powers).
Every foe is just another foe, size doesn't really matter and it should. Magic should REALLY matter, but it doesn't. Necrotic damage should be REALLY nasty and scary, but it isn't. It's just another form of damage.
There is no horror or threat or scare factor in 4E. It's flavorless because everything is so homogonized. There is no difference between unnamed damage and fire damage and necrotic damage, etc.
Granted, 3E and earlier versions had some of these same flaws as well, but not quite as bad in some areas. The goal to achieve balance in 4E, although well intended, has also trivialized and marginalized the game.
Save or die shouldn't exist. But, hit for 100 or even 150 points of damage at epic level should on occasion exist. Liches should scare the crap out of high level PCs. PCs should never consider taking on a Gargantuan Dragon without a lot of preparation. Demigods should be extra fast and scary and they should pull miracles out of their butts that startle even the most experienced PCs and players.
Enemy spell casters that bring down castles should exist on occasion. There should be serious threats. But most of the game system is "swing to hit", "do a relatively small amount of damage unless you are a Striker PC", next.
There's no mystical feel to the game mechanics. Everything is just another monster and there is no magic because everyone, PC or NPC, has powers.
It's not that larger creatures should all be Brutes. It's that larger creatures should be threatening, just because they are large. Larger creatures should be able to stampede a PC into a wall and flatten that PC. The player of that PC should be wary and cautious because of that.
Instead, larger creatures have no special abilities in these types of areas. They cannot just crush a PC, even though a Frost Giant should be able to grab a PC and rip it limb from limb or wrestle a PC to the ground with one hand. The threat is not there.
The game mechanics do not match what the imagination of the players should allow for.
Epic should truly be Epic. Mountains blowing up and Rivers being diverted and a massive monster that can flatten an entire town in minutes by just moving through it.
That doesn't happen because large creatures are no tougher than their same level smaller counterparts and magic no longer has umph to it.
All in the name of the great god Balance, all praise to his name. Amen.