One comment on path dependency. There are other factors that should be taken into account, especially when we consider such things as electric cars. In that instance, you have the oil industry, big oil lobbyists, politicians making deals, which all boils down to--at best--humans being humans, but really human greed. I remember a friend of mine in high school--eons ago--telling me about some engineer who made a car which ran on potatoes and, if memory serves, died under mysterious circumstances.
Now I'm not saying that there are conspiracies in RPGs - at least nothing on that scale. AFAIK, no one ever died for designing an RPG system that threatened "Big RPG" (aka, TSR/WotC). But there are socio-political elements - choices made based on ideology and, of course, economics (and really, ideology that serves economics).
And of course, in our personal lives, we make choices based on a variety of factors, some we don't even realized (that old psychology). To some degree this is a matter of "there's no accounting for taste" - why we like some things, but not others, or why some things we love, others hate, and how choosing the same thing, again and again, impacts our tastes. There's always that wonderful moment of discovery when you try something you previously "knew" you hated, but then decided you actually liked. But there are also previous choices that have a ripple effect. I made the choice to have children, and that has impacted everything since - probably more than any other choice I've ever made.
Around 2006ish, WotC made the choice to not only create a new edition, but one that diverged greatly from the then-current one. This was partially based on previous choices - flooding the market, so that by 2007 there was nothing new under the sun to publish or, at least, the law of diminishing returns prohibited them from publishing The Complete Book of Pages and Squires or The Blackstaff Guide to Finding a Decent Toilet in Waterdeep. But there were also other factors that influenced their decisions: the rise of video games and especially Warcraft et al, and the resulting stagnation of the D&D player base. WotC made choices in response to that; they had no choice about the context beyond their own domain.
I hate being so cynical, but in our current iteration of civilization, most of such choices comes down to economics - at least when considering the choices of businesses, especially larger ones. This is what is so fun about the RPG market beyond WotC: kickstarters and small publishers make choices all the time that aren't economically based, but are guided by love of the game. Jeffrey Talanian, the designer of the Hyperborea RPG and owner of North Wind Adventures, probably doesn't worry too much about the bottom line. He considers it, of course, but he didn't design Hyperborea to become rich. He loves his game, and publishes it, given whatever funds he has - and wants to share it with as many people as possible. This is why, for example, when I couldn't find a copy of Hyperborea 2E and inquired about it, he sent me a dinged up copy for free. That is a choice he made, that WotC would never make - and one that also has an economic impact, because it makes me want to support his company and tell others about it (so buy from North Wind, folks!).
