Marketing and business models have drastically changed in the last two to three years. Social networking and marketing is what really drives business.
For example, Daniel Solis' Do: Pilgrims of the Flying Temple has had pretty great success, and he wrote it entirely himself with just a few helpful artists. Now it is even being incorporated into curriculum in various school districts because it has such an impact on the creative process for kids.
A lot of this is simply doing what you love, and people who believe in you will follow you. Daniel built a huge following because he published a lot of small, clever games for free on his very well written and consistently maintained blog. He had a business that allowed him to work within the industry, and is already a fairly accomplished graphic designer.
I think the big thing that a lot of people aren't understanding is that D&D has set a stigma upon itself. The only people who are really familiar with RPGs already have a very strong opinion on it, either they hate it because it seems weird and nerdy, or they love it (usually because they've tried it.)
I've been going to comic book shops and places where other potential customers might wander into, and just asking them to try out OMEN for ten minutes. If they don't like it after ten minutes, they can go and eat some pizza and cookies for free. I've had about 99% of the people I've had play, even girls who get dragged along with their boyfriends, love the game. That's not tooting my own horn, that is simply a sign that the industry is so inundated with a bad reputation that it literally ruined itself.
Now when you look at how successful the concept of RPGs are, such as card games, video games, MMOs, etc, you realize that there are millions of people who would likely enjoy a TT RPG, but just aren't familiar with it or don't feel that they have the right set of variables to enjoy it.
I think that clever marketing and the correct amount of social networking is important to build the fanbase necessary to sustain a business. I'll let you know how it works out for me in the next year or two, as Sigil Stone Publishing is run entirely by friends and people with day jobs.
Anyway, if you're thinking about it, don't ever let people tell you that it can't be done or it isn't business feasible. Even the pet rock sold several million units.