We have to be careful:
Do we? What happens if we're not?

And what do we say about games like WH40K and WarMachine, which strongly mix fantasy and science fiction elements?
I think we say they don't dominate the role playing hobby in the way that fantasy does.
Also, from a technical point of view, a lot of what is labeled science fiction or fantasy is historical romance, with a thin layer of either science fiction elements or of fantasy elements added, but with those additions hardly intruding on the core story.
I don't think the answer lies in semantics! I think we all know what's meant.