I just said it was lowballed, didn't I?
When I said "flatten" I was thinking like "stomping a beer can" flatten, as in the wearer's ribcage is a thin red paste.
Size-wise, the 50lb rock would be to a giant like a baseball to a human. It is far denser than a baseball, though ... 50 lbs for a 12-foot critter would be roughly equivalent to, um ... (scribble scribble) about 7-8 pounds for a six-foot critter? Yeah, it's more like a shotput than a baseball. Anyway, my point was that the rock is a lot smaller than most people think (and yet should be deadlier than in the RAW). A human hit by a giant's thrown rock should probably be knocked down automatically.
However, to return to the original question - if the thrown rock can mangle a breastplate or shield, imagine what it would do to an unarmored person. And if it hits at an angle, it's more likely to "skip" off hard, smooth armor than soft flesh. So it seems to me that armor should provide some protection. Plus, the stereotypical Mythic Image of the Giant is "big and clumsy", so I'm not all hot and bothered if their flung rocks miss a lot. They're supposed to miss a lot.