Because the purpose of the contrast is to bridge between reader and story. In a RPG that bridge is created by the fact that I am sitting at the table with my friends while imagining myself to be part of a fantastic world.
No, the purpose of the contrast in LOTR isn't even
just to create a bridge between reader and story, but even putting that aside,
that is also useful in dnd. Halflings are
more the everyman than humans, in dnd. They like comfort just as much as you do, sitting in a comfy chair playing games with your friends, but that doesn't stop them from going on wild, terrifying, adventures.
And that's just the start of the benefit they provide to the game and the play experience.
It's a bit like playing at playing a person who is playing an adventurer. The meta seems unnecessary at best, and unstable.
It's not like that at all. There is no inherent meta, here.
In the fiction itself, the contrast between ordinary people and adventure can be established just by having ordinary people.
Putting aside that this isn't really even true, because for many, many, players and groups, what is normal in the group is normal, not what is normal amongst NPCs, I want to adress the idea that this is the purpose of halflings, more.
Because it isn't. It was the purpose of hobbits, but not even their only purpose. Hobbits and halflings exist to contrast those things both within the fellowship/party, and within the world at large. They also exist to have a people who tend toward simple lives, and show that even such folk can be heroic, when called upon. That even the simplest, smallest, folk, will rise to the need set before them. Hobbits make the
world more optimistic.
Halflings, though, are not just Hobbits. They have grown beyond that. Hell, even Hobbits aren't just what we see in the main books. Not only are there hints of more wild hobbits in the forests, and mentions of the river folk from whom hobbits originate, but games like The One Ring have extrapolated on those ideas and made whole cultures of non-shire Hobbits,
and Bree Hobbits are different from Shire Hobbits
in The Lord of The Rings.
Halflings are even more encompassing than that, including river traders with a love of adventure and new horizons and a more mobile idea of what home is, happily bustling city halflings who find niches to thrive in amongst the big folk, silent reclusive halflings with telepathy, forest dwelling halflings who live with nature and manage the danger of a very magical natural world, healer halflings and magical hostitality halflings, and plains nomadic halflings with close ties to natural spirits and the land.
What they all have in common is:
They're underestimated.
They're small.
They're community and family oriented, keeping strong bonds of family even across great divides (see the Boromar Clan of Eberron, and it's ties to the Plains) and most of them form bonds readily with non-halflings, lacking the human impulse to default to fear and distrust of outsiders. They're often cautious, but they don't assume the worst about strangers.
Most of them are fairly happy, curious, and relatively easy to please (simple pleasures and comforts), and like to make their spaces cozy and welcoming.
There are a thousand characters that spring to mind just from that, without ever getting into the specifics of each culture, their gods, their ideologies, etc.
There is value to a small underestimated race with no natural magic and who folks don't tend to distrust or dislike. A brooding assassin from that people will feel different from a human assassin, as will a friendly baker turned adventuring cleric whose healing potions are biscuits.
(That's also before we get onto the issues with reproducing, somewhat uncritically, JRRT's idealised conception of rural England. But I'll allow that that's a separate concern from the literary-type one that you replied to.)
Yeah no real issues there. It's not a problem to idealise a very broad and general style of life, like rural farm communities. Doing so is not inherently problematic. The only arguments I've ever seen against doing so have been...very classist, at best.