"What if" scenarios are always difficult, because there's a large portion of guesswork involved. But Paizo had two related advantages no other third-party creator had: a solid reputation for making good adventures (which they had gotten via Dungeon and particularly the adventure paths they had already done there), and the magazine subscriber lists which they used to gain a solid customer base from the start. In their retrospectives, they have written about what a huge advantage that was. Basically, what they did when the magazine licenses ran out was to send letters to all their subscribers telling them "Because our license has run out, the Dragon and Dungeon magazines have been cancelled with issues so-and-so being the last ones. You had a subscription that was supposed to run until issue so-and-so, and therefore we owe you $X. We can either mail you a check for that amount, or we can apply that toward a new subscription of our coming Pathfinder Adventure Paths. Which would you prefer?" And since they had such a good reputation for making adventures, a lot of their subscribers took them up on their offer. And this was all long before there were any plans for making the Pathfinder RPG.
They've also written about what a huge gamble the Pathfinder RPG was. The main reasons they made it in the first place were (a) they didn't care for the 4e rules and didn't want to make adventures for them, and (b) the GSL had many provisions that made it intolerable for them to use. Had those things not been true, I'm sure they would have preferred to make Pathfinder APs for a hypothetical different 4e rather than risking making a new(ish) RPG. And had they done so, they'd have made them for the whole D&D market rather than the portion that decided to reject 4e. Would that have worked out better or worse for them in the long run? It's really impossible to say.
