We agree on that much.
This is the fundamental point of disagreement. Certainty, I think if the relationship between humans and elves was meaningfully analogous to that between humans and Neanderthals, species would be a perfectly cromulent way to describe them. But I do not think it is meaningfully analogous, because of the point above on which we agree.
I don’t think it needs to absolutely hold up in every circumstance, I just think it needs to adequately describe what these groups
are. They aren’t the furthest ends of the branches of a tree of life, they’re groupings of traits a character inherits from one or both of their parents. As
@Cadence observed earlier, it’s more Lamarckian than Darwinian. I would call them heritages, or ancestries, or parentages, or something to that effect, communicating that the relevant factor is descent from parent to child, not phylogenic category.