Which is why they mad ethe product
It's why they cloned the cover of the Red Box for Essential, too, but that went over like a lead balloon. Timing is everything. And, with a nerd-beloved franchise like D&D, walking that tight-rope between fan-acceptable and mainstream-accessible is critical. 5e erred on the side of acceptability, with good results.
the Keep on the Shadowfell set and the Essentials box, too, and those did not have the staying power of the Lost Mines of Phandelver.
You mean the starter set, since the module in question was just in it, not being sold separately. I don't even recall if KotSf was physically in a starter set, or not (again, having not cracked open a starter set since 1980), but the Essentials Red Box was prettymuch redundant to the other Essentials offerings (which, heck, had a certain level of redundancy in HotFL/HotFK, which shared a lot more than just the first 4 letters of their goofy-sounding acronyms). Starter Sets just weren't that necessary, the game was more accessible to new players in that period, especially if they just walked in and started playing Encounters, they were a lot more likely to pick up a PH or a DDI subscription than a 'try the game for the first time' set for a game they'd just tried for the first time. ::shrug:: And it certainly was no tragedy when KotSf left print - the tragedy was it stayed up on line to afflict more players with it's suckiness. ;P
Don't know too much about the Basic Set years, as both B/X and BECMI were released before I was born.
In the fad era, no PH sold half what the Basic Set did. Of course, the two-prong strategy and the Expert Set probably contributed to that, as well, but it was just a very baroque and inaccessible game. Later eds, PHs were the top sellers, if there even was a basic set, because they were selling mainly to established players, the mainstream having lost all interest and virtually all awareness of D&D.
Oh, I should mention: D&D in the fad years was selling 3 different versions. The old 0D&D booklets were still being re-printed (and an unauthorized clone, Arduin Grimoire was out there, as well). The Basic Set was meant to lead into the Expert Set (B/X) and later the rest of BECMI as it was released, while, at the same time, Advanced D&D had been published since '77 (complete since '79 - you think 5e releases are slow, 1e started out going a book a year, even for the core 3!), and lots of people seemed to assume that the path was Basic -> Advanced, not Basic -> B(X/E)CMI. Supplementary products weren't that carefully labeled, either. So it was an age of chaos, really.
In spite of that bizarre shelf presence, the fad still fadded along with great success.
But, yeah, the Basic Set, as the obvious starting point for the curious, moved the most units, fad or come-back.