Well, for the same reason that, by and large, most alien races look surprisingly similar - it makes them distinct. There's a reason that you don't see portly Vulcans or short skinny Klingons or whatever. By typecasting a given alien race, you provide a hook for play. Short, stout, likes good food, is a decent enough hook to start from. It lets everyone at the table know that you're playing a halfling.
That's a good point. Also, (because I'm a freak for "realism" (I know, shoot me (I know, too many brackets (sue me)))) it is true that if you exactly mirror the diversity of human shapes, then the race seems like humans*. If one were to want to do it right, one would pick a few characteristics that won't vary much within the halfling, and a few other characteristics that will vary more in halflings than they do in humans. So the fantasy race would still have its diversity, but in a distinct way from humans. But that's advanced fantasy art theory, and more than I can expect from D&D, I think.
*Speaking of the race seeming like humans though, I'm okay with this for halflings, because Tolkien explained that halflings are related to humans. I don't have a huge problem with halflings essentially being small people.
Giving halflings the "small humans" niche allows a conceptual space for gnomes to be more magical, whimsical, and distinct in their own right.
Look at elves. My main beef with elves is that they don't really ever have much of a hook other than poncy. They aren't distinguished enough and players, IME, tend to play elves as slightly longer lived humans with better vision.
That is true. I used to hate the extreme long-ears elves, but I'd rather have that than the humans-wearing-Spock-ears that appeared in 3E and 4E. At least the long ears is a distinction.
I would love to see artwork in the new edition that presents a wide variety of what elves could look like. Long ears, short ears, tall, short, feline features. I think that's what Jeremy meant when he said they took 4E too seriously. 4E had a very set way of expressing each race. The tieflings bothered me especially, how they all had the same horns. I had always depicted tieflings as being extremely varied in how their fiendish blood manifested itself (often not physically at all), but 4E sort of said "this is what tieflings look like now.
Same deal with orcs. There are so many ways to draw orcs floating around the world of fantasy, it would be a pity to pick only one and say that that's the D&D orc. Maybe some orcs have horns. Maybe some are green, some are grey, and some are black. Maybe some just look like humans.
There is this temptation also to create flavour and mechanics for all the different artistic looks, eg. making the "horned orc" a subrace. And I'd be happy if they did that, but I'd be just as happy if they didn't mention it at all, and just sometimes drew one kind of hobbit, and other times drew the other kind of hobbit. It might be nice to leave this kind of thing as an exercise for the budding Dungeon Master.