My own view - grounded in both theoretical analysis (especially on The Forge) and play experience - is that mechanics can make a huge difference to how hard or easy it is for story to emerge from play.
Of course.
By "mechanics" here I mean at least (i) PC build and advancement rules, (ii) action resolution rules, and (iii) encounter-building guidelines.
Both mechanics, and the mechanical design and presentation of story elements, can make is significantly easier or harder to make encounters more interesting. For example, in Oriental Adventures (the mid-80s original), PC building includes giving your PC a family (with a history) and honour. This history and honour mirrors a default cosmology and mythos, which in turn is embodied in many of the monsters in the game. It is much easier to pick up OA and design an encounter that will engage your players (via their PCs) than it is to do so with the core AD&D books.
Action resolution mechanics can also matter here. For example, the more the action resolution mechanics draw attention and effort at the table away from stakes and theme, and onto accounting and measuring and bickering over the details of exploration, the harder they make it for the GM to adjudicate encounters in a way that encourages the emergence of a story.
It really depends on how you're trying to achieve a story. "Bickering over the details of exploration" can lead to situations in the game where an "interesting" story
emerges from play. This may not suit your wants, but this sort of emergent story play is essential for the majority of my enjoyment when I run a game. I want to see how the mechanics of the game cause the story to
emerge and
evolve. Passing or failing each obstacle has an effect on the story, and I like seeing the results.
Is it the same type of "interesting story" that you can achieve by focusing on stakes and theme? Probably not. I think my preference is clear, but it's just a preference.
Also, I should note that while 4e doesn't fight the "stakes and theme" approach to producing an "interesting story", I don't know of many mechanics which push it wildly ahead in this area compared to games with more focused mechanics (I'll comment more on this below).
4e isn't HeroWars/Quest or Burning Wheel, but it's not B/X D&d either. It's like the original OA, but richer.
But players, by choosing race, class, paragon path, epic destiny, powers and feats get to put elements of the backstory into play, and shape it in various ways, that is noticeable and (in my experience) significant. And these player choices all shape and interact with GM choices, because the story elements are mostly integrated into the same mythos, history and associated themes and stakes.
With a little twisting, I think that most people would be about as satisfied with 4e and with previous editions when it comes to this approach. That is, I think that 4e is a better vehicle for this style of play, but it's not inherently pushing for it. And others, who aren't so intellectually aware of this sort of play, may be quite satisfied in past editions picking "elf" and being involved in elf-things and elf-relationships.
Again, I don't think 4e works against this sort of play, and probably allows it easier than previous editions of D&D. However, I also don't think that it is inherently supported by the mechanics (something we probably disagree on). To me, producing an "interesting story" has much more to do with GM fiat than with any rules in any edition of the game. The GM still needs to deal with what is being interacted with (your "framing interesting encounters" or the like, and my "exploring the evolving setting"), or else we're just dealing with mechanics in a void.
Now, obviously, 4e can use its mechanics to give narrative control to its players. You even extend that to skill challenges and not solely powers, for example. However, the skill challenge mechanics themselves don't
support this style of play, even if they don't fight against it. They weren't designed, in my opinion, with the goal of player narrative control. They were designed to keep everyone involved and useful during (mostly) non-combat events, and to give a solid framework for adjudicating success or failure in those events (with complications arising and the fiction moving forward as a result).
And, while that's a useful tool, it's not like Neonchameleon's disguise power from Spirit of the Century: your PC disappears, and you can later declare "nope, that guy was actually me all along!"
That, in my opinion, is a mechanic that is designed to give players more narrative control over the game. In my opinion, 4e doesn't fight
against the style of play you describe (and enjoy), it just wasn't designed to
support it. That is, in Spirit of the Century, you
have to utilize player narrative control to use the ability; there's no way around it. In 4e, there's no such focused mechanic when it comes to resolving skill challenges. That is, skill challenges can be framed as either "here's our story goal, and if we succeed we cause the story to happen a certain way" (on a success I cause the story to be shaped this way), or they can be "here's my goal in-game, let's see if I can achieve it" (on a success I'm closer to achieving my goal in-game; what's the story look like now, GM?).
You can't escape the forced use of narrative control in Spirit of the Century, but you can in 4e, because the mechanics don't dictate that style of play. That opens up more play styles when using the system, but definitely moves you further from strong, deliberate player narrative control over the game's "interesting story" section. My take on it, at least. As always, play what you like
Exactly. More narrative than D&D had been in the past, still no where near as focused on the narrative as other games out there. Not really a 'narrativist game,' but 'narrativist for D&D.' And, a big part of that is what it models - it doesn't model, say "what a character might be able to realistically do in a universe where magic exists" but "what characters actually do in heroic (almost left that out again) fantasy stories, no matter how crazy those things may seem."
I don't know if
that is actually a player narrative control thing. It's still an attempt to tell an "interesting fantasy story", it just a different method from "what a character might be able to realistically do in a universe where magic exists." Giving the players (not PCs) the ability to exercise an ability at will (not meaning all the time, but when they choose) is more a type of granting player narrative control, I think.
It's just a matter of degree. 4e is more in the narrative direction than prior eds. I think there were some design decisions that could only have been made the way they were if that was a conscious choice (the new way classes were handled, and the return to monsters being handled differently than PCs, for instance). And there are quite a few places where the rules will talk about 'moving the story along' or 'moving the game along,' particularly in resolution systems like skill challenges and p42 'say yes' improvisation. I don't think narrativism is the full thrust of 4e, but 'realism' is definitely out as a major design consideration.
Certainly, things like minions and solos are an attempt to model the genre, and not in-world physics. No disagreement from me there. The "say yes" is purely advice, to my knowledge, and not baked into the mechanics themselves. If we're talking granting strong, deliberate player narrative control, see my comment to pemerton in this post about the disguise ability Neonchameleon posted.
It's also , by dint of being balanced, whether it was intentional or not, a lot more even-handed about which PCs have the potential for some player narrative influence. That does, I think, make it better for the 'shared storytelling' aspect. But, then, the balance, alone, helps there. :shrug:
Balance does indeed help in this regard, but everyone contributing all areas has its drawbacks, too. It makes it harder to shine in a "only I can do this" kind of way. People like both, and it's just preference, obviously. I personally quite like balance, though. As always, play what you like
