You know, I mentioned Green Ronin's brilliant game Spaceship Zero earlier tonight, and someone said "it's a shame it didn't get more support." The same complaint was heard with Call of Cthulhu d20, a book that was designed to be a complete unit. There's a perceived vigor in a product line that continues to publish supplements, and I think that people are reluctant to buy from "dead" product lines.
I was the poster who lamented that Spaceship Zero got no support. I thought the game was good, and had there been supplements in the stores, I'd have purchased them.
It had nothing to do with "dead" product lines- I simply will not buy .pdfs.
That said, other than the "supplement treadmill" I can only think of a few ways to keep an RPG game companies in business (and each has its own problems):
1) Publish new games. NASCAR RPG anyone?
2) Publish new settings for old rulesets. Technically, these are not supplements, and would probably sell almost as well as supplements if each setting introduced modular rules tweeks (as in new feats, new races, new spells). In this world, not only would WotC have produced Greyhawk, FR, DL, Eberron, Rokugan, Ravenloft, Maztica, etc., you'd also see them do stuff from their other games (like Dominaria, Everwhere, or Primal Order) and licenced products from extant fiction (Conan? LotR? Deryini?).
3) Publish rules conversion books (either from old rules to new, or other game systems to yours & back again).
4) Publish a "house-organ" magazine.
5) Publish an industry-wide magazine.
6) Produce game aids: maps, cardboard minis/chits (SJG's Cardboard Heroes), metal or plastic minis, Character background generators (TFG's Central Casting) and the like.
7) Make something collectible, especially if it ties into one of your core games. Imagine, if you will, that WotC published no spells in the PHB, but instead made the spells only available as collectible cards...(WotC, I hope your 4Ed designers are NOT listening)
8) Make your product just fragile enough to need replacing every few years. (Booooo! Hisssss!)
9) Produce spin-off products. Space: 1889 had a boardgame, Sky Galleons of Mars. Going the other direction, Battletech spawned MechWarrior. Many games have minis combat games associated with them, but it's not a universally viable option. Some games have moved into the MMORPG world, though more often than not, they've been preceeded by computer games inspired by their core product. Some (unfortunately) have even had movies made based on their games.
10) Publish rules updates/revisions/new editions on a relatively regular schedule.