And where are they now? Yes, there are a few % of their DNA within the species still in some populations, but...gone. And consider the tech level, the population level, of those earlier hominid's. Their ability to project power, to grow their populations, to wage war.
Now, instead, imagine its pseudo-medieval, and we have magic, and opposing gods. Look at the sad history of our world, and the various cultures within, when there is a power disparity.
In actual human history, yes, when the triumph our our species ancestors was to be slightly beyond the reach of the beasts of the world, separate groups may or may not have lived near eachother, and may or may not, have wiped eachother out.
Not exactly comparable to the various races of our Fantasy worlds, to me.