I think you might have missed something....
You see, the thing with elves and gnomes is that they are inherently magical. The problem is, they also inhabit areas that are a bit too familiar. Everyone can go there and we are well aware of the dangers. Elves come from the forest.
Humans? Well, humans are descended from apes that live in forests. Sure, our branch came out of forests onto plains and were likely semi-aquatic to reach our modern species... but plenty of other kinds of hominids lived primarily in forests. And our species never really moved all that far from them, depending very much upon them until farming was invented and we could take better advantage of plains.
So to say that people are inherently different or special or magical simply because they live in a place that is easily accessible to us, plenty of us have always lived in and is very much familiar-- that is an odd preposition. The only way to make it work is to assert that they come from a very special sort of forest, a sort we cannot go to-- another world deep, deep within the forest.
But Dwarves? Sure, humans have always lived in caves to an extent... but even "cavemen" didn't live that deep underground. And we have had miners since before recorded history, but a lot of special allowances had to be made for them. But to actually, truly, live one's entire life deep in the recesses of the Earth that general humans could only reach with special training and special allowances and even then are all too likely to die down there in a matter of hours...
Well, that is already inherently special. You don't need to make it any more special. Just like Merpeople living in the ocean or a race that might live in frigid snowlands... you don't need to make it an extra special version of an already entirely inhospitable environment for humans.
Then again, I do suppose that given Dungeons and Dragons revolves so much around Dungeon Crawling-- i.e. occupying those depths for days, even weeks on end with the entire comprehension of the actual danger and threat and paranoia being exposed to such an environment long term would actually have on a typical person... and Dwarves have been so very normalized, I guess I could understand why you would want to have them originating from an extra special magical version of this environment. The whole idea of a people who willingly, knowingly live in a claustrophobic environment of eternal darkness with possible pits dropping dozens, if not hundreds, of feet around any given random unfamiliar corridor and could all so very easily be flooded by any random rainstorm... where no plants can grow, no natural animals of decent size could dwell, where it would be natural for creatures adapted to the environment to become albino and blind....
Reflecting on it, truly, for any length of time makes it utterly ridiculous that Dwarves are so entirely passably "typical European human" as they are. They are at best a couple dozen generations removed from humans, not thousands of years of evolutionary (particularly magically-accelerated magical evolutionary) adaptation. They don't even pass as a species that has actually adapted and acclimated to the supposed environment they are meant to be most comfortable in-- they most certainly do not originate from an extra special magically exaggerated version of that environment.
