Planescape and Spelljammer, and I know this will sound more pretentious than it is but this is still a pretentious statement, were in no way mainstream ideas in the past, and were much more likely to resound with people who would write in one of their top 3 traits "creative." Or, in other words, a lot of people who were happy with the "box" of standard D&D were very unlikely to care about hyper-niche D&D, even if it still had elves and dwarves and what not in it. In fact, this seems to also prove that having elves and dwarves in Dark Sun really didn't help it all that much, if at all.
Meanwhile, people who prefer to approach D&D as a canvas for a lot of weird ideas they have or similar concepts likely resonated very strongly with the non-trad settings of Planescape, Spelljammer, and Dark Sun. These "outside the box" fantasy ideas showed them more ways to do things they had never imagined, whereas most consumers (and I'm pulling this all straight out of my ass) likely are just happy making stories/experiences with just the tools given, no more and no less.
I am not saying either crowd is better than the other, but this is what the data + so many people's unique perspectives seems to be telling me.
That said, still surprised that Spelljammer outsold Planescape. Kind of makes sense though. Fantasy -> Space -> Weird Mysticism probably tracks to how mainstream tastes are in movies, shows, books, comics, etc too.