Growing up, my impression was that Dungeons and Dragons, Call of Cthulhu, and Vampire the Masquerade were the "big three" games. D&D is quite obviously alive, and Cthuhlu is still around, but I don't see much going on with Vampire or the other World of Darkness games.
Am I mistaken to think that they fell into decline? If not, what do you think happened to make them fizzle out?
Not so much fizzled as reached the logical conclusion of its publishing model.
The World of Darkness was selling roleplaying in a World of Darkness that got Bigger, Edgier, and Grimderper with each supplement. It was the setting they were selling, not the rules - to the point where the rules lead to a different playstyle from the games, and they'd accuse anyone who wanted to use what the rules actually did of "Rollplaying, not roleplaying". (Incidentally, this is what triggered The Forge - wanting games that did what they said on the tin). To maintain interest, with almost every supplement the metaplot and the NPCs had to be Bigger! Edgier! and Grimderper! than last time. There's only so many sudden-yet-inevitable-betrayals from Saulot and times you can nuke Antediluvians before the whole thing collapses in ridiculousness.
Which meant that White Wolf needed a new world. A new World of Darkness. They came out with it in 2004 and gave it better rules than the oWoD (which were not a selling point given that for a decade and a half they had been telling people to ignore the rules). It also had a cut down metaplot (which again was an improvement but not a selling point as the only people still buying White Wolf then were buying it
for the metaplot). The stakes were lowered to personal horror, and the richness of the world - they lost however many supplements. This meant that the reaction to the nWoD was a general "meh" and edition wars that made 3e/4e seem friendly.
That said, there's some interesting material coming out right now for what is effectively WoD3 -
the revised Vampire is getting back to its horror roots and came out just over a month ago, and
Demon: The Descent was kickstarted a couple of months ago and comes out later this year. The new
Convention: Void Engineers is excellent and less than six months old, but to illustrate how badly the nWoD did, the first nMage convention book
came out in 2005 -
Progenitors,
Syndicate, and Void Engineers came out less than a year ago.