enh? Rapidly ecclipsed? You'll need to refresh my memory. I don't recall anything published I'd call narrative until....jeez, maybe not 'til after 2000. There was a weird little super-hero game called Panels, which was apparently around in 1999. (It was so simple it was basically somebody's geocities page. Doesn't exist now, AFAICT.) Panels was the first system I ever saw that really made me take notice of its narrative possibilities.
Goodness, you were closeted

Amber Diceless springs instantly to mind, published in 1991. Many other games are older than you seem to think. Toon?! --all meta-game. TSR itself incorporated narrative elements in Top Secret SI. Another game which had them was that game about being a mobster in Chicago in the 1930's, whatever it was called. En Garde! also has many story-telling elements and it was published in 1975 (and fairly represents a line of RPG development entirely unrelated to D&D) which was quite well known in the day and is still in print today. Sure, I agree that game design was less advanced, but I think you sell 90's game design a little short.
I do remember a whole lotta systems in the 90's that did the same thing 2e did...talked all about story, but did nothing to support it. Heck, WW/WoD talked about it lot (even named their system "Storyteller"), but still didn't have anything significant mechanically. So much so that some credit them with "turning off" a lot of would-be narrative gamers. AFAICT, the 90's was the age of "resolution experimentation". Everybody seemed to think that they could fix everything if they just got the game's resolution system right...'cause somehow the other 999 ways to roll dice were missing something.
I agree about 2e's intentions, though. You could see Grubb wanted to make something with the story at the heart, but the tools weren't there. IMO, 2e's abject failure in this regard is what spawned the Forge and its thinking, but it took about a decade before anybody actually made much progress figuring out how to do narrative mechanics.
Eh, the problem was 2e did nothing. It neither cleaned up and rationalized 1e's rules, broke any new ground, etc. In fact for a good long time we didn't even bother with it because why spend money on new books (and 1e core books last FOREVER). Surely I can't say what Jeff did and didn't know about plot-based mechanics. What was not super well understood at the time wasn't that such mechanics existed, but more exactly how to employ them in a game design. All I can say is 2e sorely disappointed us at the time, it seemed like more of the same. We wanted mechanics like 'hero points' or something to get rid of the depressingly conservative play that glass D&D characters engendered. I recall having these discussions with some of my DM friends.
That's certainly an interesting take on TSR's problems. Obviously they had some, but that's a unique interpretation of what they were. I can't say too much about 1e's peak popularity, but I would guess more people were playing 2e at its peak, just from the growth of the hobby as a whole. Certainly in the places I was gaming 1e was all but replaced, if only because of a lack of books. Plenty of people I knew played material from both editions side by side, though, so that's hard to judge. (Whaddya call a game where the DM is using a BECMI adventure with a 1e MM and most of the players are using 2e characters, but one guy has a 1e Barbarian?)
Well, I recall statements made about 2e's sales being something like 1/2 what 1e's core book sales were. I'm not so sure the hobby GREW from 1983 to say 1993 either. It became a good bit lower profile after the days of the crazy RP hating nuts and the D&D cartoon. OTOH 2e was in bookstores (albeit the product displays appeared to gather much dust). Yes, we all played 1e and 2e material together, they were the same game mechanically, with a couple of fairly trivial exceptions (slight changes in AC, monsters tended to be beefed up). We all have different experiences of course, but IME everyone was playing in the early 80's, heck in 1980 I literally moved to a new town and the FIRST PERSON I MET randomly on the street was another DM, and I had a game going at school in like the first 2 hours. I think the mid-90's were more a time when those of us who kept playing were in our 30's now and could afford to buy a lot of stuff. I'm not at all sure the absolute numbers ever hit the 1980's. Maybe that wasn't much to do with 2e realistically, but I don't think 2e helped any. It opened up the window for other companies to establish real inroads into the market.
WW almost outsold TSR there for a couple years. Maybe their game wasn't exactly mechanically a game with narrative support, but people clearly saw it as a game that was trying to promote RP as opposed to whatever 2e was doing, dungeon crawls.