Possibly D&D has smudged out their original purpose. Halflings are the crofters of old England. Whereas elves capture the arts-and-crafts movement. Tolkien's faith in the deep roots and good hearts of those people, and fears about industrialization, made them the perfect foils in his epic. The corruption of the Shire more than anything else showed the worst of what might happen. In particular, the corruption of individual halflings who went along with it. Possibly Tom Bombadil represents a kind of arch-halfling, so grounded as to be unmovable by the artifices of industrialists. The elves on the other hand were tricked when industry crafted things that looked like art.And this takes me to halflings. What's their niche? Short-person. Are they the only race in that niche? Only if you don't count gnomes, dwarves, kobolds, and goblins (and Fairies if you count UA, and I'm not even counting the Lineages/Races that can be small or medium, including Verdan). Are they strongly rooted in the identity of most worlds that they're included in? Not really. If you take Halflings out of the Forgotten Realms or Exandria, it doesn't really change anything important/major about the settings. If you remove them from Dark Sun you don't have cannibal halflings, which are a cool tidbit about the setting, but certainly not essential to its identity, IMO. Eberron probably changes the most noticeably of any of these listed settings, as it has Talenta Plains, Dragonmarked, and House Boromar Halflings, but even then, you could just as easily replace all halflings with Gnomes (or possibly even Goblins) and get practically the same outcome. What is their lore-based reason to exist in most D&D world's? There's rarely actually ever one of these, and even if there is, the explanation is lacking (cause this god I just came up with to create halflings created halflings), and/or could just be summed up by "Halflings are in this world because they exist in D&D". And why do Halflings exist in D&D as a whole? Because Tolkien's works (a huge part of the inspiration of D&D) included Hobbits.
And that's where the issue (for me) comes down to. Their existence is circular. They exist for no real narrative or plot-driving purposes, but because Lord of the Rings and the Hobbit had small-folk as a race for some of its most prominent characters. And that's not a "bad" reason to warrant their existence in a fantasy game where quite literally anything can exist, but it's just not a "good" one, either (and by "a good reason to warrant existing", I meant it as in a reason that empowers creative thought, drives/inspires plot points, and motivates players to think a bit more about the identity of their characters). Warforged exist for a good reason (to provoke discussion and tropes of "what measure is a non-human") and give a lot of inspiration for both character backstory and plot points. Felshen exist in my D&D world to create plot points about the Felyik Conflict (shorthand for Felshen-Yikkan Conflict/Wars), to give players ideas on how their character(s) feel about major parts of the world (the magical goblinoid and psionic humanoid societies), and to drive discussion on who the "good" and the "bad" in the conflict are (it's neither, all shades of gray, but some individuals and mindsets are more wrong or right than others). The Kryn Dynasty exists in Exandria to drive discussion on essentially the same issue as Paarthurnax's famous question of "What is better - to be born good, or to overcome your evil nature through great effort?" The Warforged, the Felshen, the Kryn Dynasty, (and endless further examples), all exist for what I define as "good" reasons. They exist for story-driving reasons, while Halflings just exist to be "short people that are humans . . . but short".
So if you want a grounded people, who are hearty and at heart kind and honest, then you have halflings. They care for crafted things, but not so much art (which they respect, but it is above them). Well made, useful things. Their purpose in a campaign is as what is to be protected. They are the good children of the home counties. There is much wrong already with this whole picture (!), of course. But anyway, their sense for good and fair is so deep that at a pinch you know they will always do the right thing. Even if they don't know the way.
This is a coherent purpose, although its value will depend on the game world in question, and what is going on there.