I posted a longer discussion about this recently, but in short, there is a fundamental mistake in comparing 3E to 4E here.
I think I basically agree with you.
I posted upthread that, when 4e was released, I assumed that WotC had the market research to confirm Ron Edwards' specualtion that a well-supported RPG that departed from "simulationism-by-habit" could really take off.
It seems, however, that WotC were taking a bit of a punt, and that Edwards' speculation was wrong. (Although I think WotC didn't do as good a job as they might have in writing their rules texts - see below for more on this.)
In some ways it seems that 4E advocates a sandbox approach to the game, yet doesn't offer enough tools and support for running a sandbox-style game. So in the end 4E has been in a kind of in-between state, between its original "points of light" sandbox approach and its half-baked settings.
See, I don't see why they can't do both - advocate a sandbox approach, provide support for running a sandbox game, but also create and support and ongoing setting (like Nentir Vale).
My response to this - and it's just my intuition, it's not coming from any deep insight into either RPG business or RPG design - is twofold.
On "why not both" - I think it's actually a bit of a challenge to come up with action resolution mechanics that suit both "just in time" GMing of a situation-driven game, and that suit "world/story" GMing of the sort that a developed setting supports.
I'm not saying it's impossible - HeroWars, for example, is a game that tries to combine both approaches using Glorantha as the gameworld.
But just one example as to why it might be tricky - in a "world/story" game, the GM is likely to know the obstacles in advance, and to present them in some detail to the players, and the players will then be looking for action resolution mechanics that really let them enage with the detail of those challenges. And those action resolution mecanics have to produce results that put the players on the same page as the GM - otherwise the game won't run smoothly.
On the other hand, in a "just in time" game the GM is more likely to be adding details to a situation in response to ideas and interest expressed by the players as play is going on. So the action resolution mechanics have to be ones that encourage the players to produce those sorts of ideas, and that let them pursue their interests - otherwise the GM will be left with nothing to build on.
Skill challenges are, in my view, a good attempt at a mechanic for the second sort of play - and that is how the
rules for skill challenges are presented in the DMG and PHB (I can provide quotes if desired). But skill challenges are a fairly poor mechanic for the first sort of play - they tend to produce the "exercise in dice rolling" experience, as the GM describes the situation to the players, and tells them their options, and the players roll the dice. And this is how the
examples of skill challenges both in the DMG and in the WotC adventures have tended to be experienced (not by everyone, but I think at least by a majority of the posts I've read on these forums).
Second response: I think Ron Edwards is right when he says that authors of non-simulationsist RPGs mechanics are often afraid to explain, in plain language, how they intend their mechanics to be used. They fall back into the language of simulationist RPGs. And this makes the rulebooks for their games at least moderately incoherent. And in my view 4e has this problem. (Worlds and Monsters is an honourable exception, but its candidness about the way in which monsters and other game elements are intended, by the designers, to be used by a GM in running adventures is reflected in only one part of the core 4e rules that I can recall - namely, in the DMG's brief discussion of languages. EDIT TO THIS: of course the DMG makes it very clear how monsters are to be used in combat encounter design and resolution - but I'm talking about the use of game elements to create an FRPG experience - indeed, the fact that the DMG goes metagame
only in relation to combat, but not in relation to GMing overall is part of the problem.)
When I look at the rules in a book like Hubris's Maelstrom Storytelling, or Robin Laws HeroQuest II - which are both sterling exceptions to Edwards' generalisation about non-simulationist game texts - and compare them to WotC's efforts, it makes me cry (well, not literally!). If only WotC had actually explained to readers of the rulebooks
how the sort of game that the 4e mechanics support is played and GMed, maybe 4e would not have so easily fallen victim to the "dice rolling"/"minis game"/"WoW" critiques. Instead WotC left this as an exercise for the reader - and those who tried to play the game in the typical sort of way that 2nd ed AD&D or 3E was played had, I assume, a fairly mediocre experience, of rolling a few dice and making a few tactical decisions but not really experiencing the evocative power of gaming in a fantasy world.
But like I said upthread, and earlier in this post in response to BryonD, maybe the sort of game that 4e exemplifies is just not going to be popular in any event. In which case I fully agree with you that the problem for 4e's popularity is the setting issue,
but precisely because this is (in my view) a symptom of deeper features of the mechanics which it turns out many RPGers seem not to want.
