But (this is a genuine question, so please take it in good faith), what hooks them about the non-human characters,
That would depend entirely on the individual. You'd have to ask each person seperately. I really like the planetouched people--tiefs, aasimar, genasi, etc. Maybe it
is because I'm LGBT+ (I'm ace) or maybe it's because I'm autistic--because of both these things, I've managed to feel "other" for most of my life and could related to them. Maybe it was purely because I love DiTerlizzi's art and 2e Planescape.
I have two friends who love playing catfolk. One just really likes cats; the other likes cats and has a whole otherkin thing going on. Both of them get kind of upset if they don't have that option for a character, even though the first tends to play their tabaxi like stereotypical housecats. I have another friend who tends to gravitate to the burly races--orcs, dwarfs, warforged. I don't know why exactly, but since he got annoyed once when I was talking about wanting more gracile warforged (since I don't like how clunky the art normally makes them look), I have to assume he likes the burlyness and solidity of those races, and a fat elf may be burly but wouldn't scratch that itch.
f it's not distinctive mechanics and it's not distinctive appearance? If it's the fictional culture or society, surely that varies from campaign to campaign? (And you seem to imply that most gamers don't really care about the fictional society with your comment about "bearded Scottish dwarves" anyway.)
To start off with, most races
do have distinctive mechanics. For instance, a hill dwarf has darkvision, dwarven resilience, combat training, tool proficiency, stonecunning, and dwarven toughness, while a wood elf has darkvision, keen senses, fey ancestry, trance, elf weapon training, fleet of foot, and mask of the wild. Other than darkvision--and that's a whole 'nother conversaion--they're entirely different.
But no, I don't think most people really bother with creating fully-fledged cultures for each race/heritage/ancestry in their game. Nor do most players really want to remember more than a few bullet points. Xenofiction is
hard, after all, and it can be difficult to come up with nonhuman cultural elements that are easy to remember
and will be fully playable by humans
and won't be skeevy
and won't involve more worldbuilding than most DMs have the
time to put in. How many settings really take into consideration what the presence of magic, monsters, and active gods would really
do to a world? Throw in truly inhuman non-humans, and it would be overwhelming for a lot of players.
I guess there are still things like dragonborn or maybe orcs or tieflings, but should humans, elves, dwarves, halflings, and gnomes just be lumped into "humanoids" at this point? And do we lose anything if we do that?
Well, ignoring how, as I showed you above, different the humanoid races are... lots of games don't bother with racial mechanics, or treat them as no different than any other type of mechanic, or have the differences as an option. So, for D&D, probably not. But plenty of other games have already done this.
