This connection intrigues me a bit because it's not something that has really occurred to me. Perhaps it is your more philosophical bent that is neutralizing the possibility space here, but I don't see an opposition there.
<snip>
I think you may be conflating the ability to change the multiverse with "wish fulfillment." Imposing your beliefs on the world should never be an easy or straightforward process.
You're correct that I'm equating, or at least linking (I'll dispute "conflating", because that implies my equation/linkage is mistaken!), "beliefs shape the world" with "wish fullfilment".
I had a supervision yesterday afternoon with a PhD student of mine writing her thesis on social possibility as a factor in collective political action - ie what sort of social transformations are feasible, and convesely what sorts of social facts have to be treated as limits on possibility in much the same way as facts about gravity or the refractive index of water. The same issue, about wish fulfillment, arises in a very pointed way in this work.
For instance, Merlau-Ponty, after the liberation of France, wrote an essay (the name of which escapes me - I know it primarily through my student's commentry on it) in which he argued that the collaborators (whom he wanted shot) had acquiesced in the face of "realism", whereas the resistance (with whom he identified himself) had "answered to reasons that had not yet come to be" (loose paraphrase) - that is, by acting, and by acting successfully against all sense of realism, they created a future in which Nazism had lost, and hence which generated reasons - ie reasons not to acquiesce and compromise in the reality of Nazism - which retroactively gave validity to their choice to resist.
Now when Merleau-Ponty says this it tends to look somewhat romantic and heroic. But when an aide to President Bush Jr made a similar remark in the more recent past, dismissing certain political commentators for being stuck in the
"reality-based community", some people thought that this dismissal, and the corresponding assertion that "we create our own reality", was both hubristic and self-delusory.
I tend to think that, in an RPG, where it is already quite hard to generate subtle emotional responses (due to the nature of the medium and typical participants), an idea that belief creates reality is going to tend towards "wish fulfillment" ie the sorts of nuances that are in play in our responses to Merleau-Ponty and the Bush administration won't be enlivened very easily. Hence what seem like they might, prima-facie, be "horrors" or "moral costs" will be retrospectively "believed" away without emotional cost.
This is an intuition of mine. It's got no more (nor less) validity than anyone else's intuition based on extensive RPGing experience.
Plenty of narrative fiction is also a puzzle (obligatory James Joyce allusion for the day complete!). And every puzzle has a narrative core (I begin. I confront obstacles. I get better. I succeed....or I fail...).
Narrative is a part of all those modules (even if it's just the narrative of how Joe the Fighter failed to check for traps before he died)
<snip>
Narrative fiction can provide very similar entertainment to crossword puzzles and board games in that they allow for problem-solving and deduction and solution-seeking. Heck, the entire genre of crime novels -- one of the most popular book genres! -- is built around evoking that emotion. I've never heard a description of a James Bond movie that accused it of not having a dramatic conflict. In terms of its emotional intensity, the good ones at least have very dramatic conflict!
Part of the issue here is that there are no canonical terms in RPG discussions - Ron Edwards tried to coin some but they have issues of their own, and in any event aren't widely used on ENworld (in the sense that he uses them).
Robin Laws has terms, too - some of my thinking in this thread is influenced by his Hamlet's Hit Points, which I bought mosty to read Robin Laws' take on Casablanca - but likewise they are not widely shared in RPG discussions.
Drama
What counts as "dramatic" is very flexible - sometimes it is just a synonym for "exciting". James Bond movies are exciting, but they don't involve drama in a more narrow sense of
emotionally moving confict. The characters personalities and commitments, for instance, don't change. Their values are not placed under pressure. Bond never tries to show Goldfinger the error of his ways - in fact, as is shown by the near-mandatory meeting between Bond and each antagonist, as personalities they are almost identical (resourceful, ruthless, clever) but they just happen to be fighting for opposed sides.
Detective fiction, at least of the canonical "whodunnit" variety, doesn't involve this sort of dramatic conflict either. Raymond Chandler and Dashiell Hammett are different in this respect, but then they're not canonical "whodunnits". In The Big Sleep, for instance (and admittedly here I'm working more from the movie than the book, which I've not read for a very long time),
does see Marlowe's values put under pressure, for instance in his relationship to the members of the family that hire him.
But in canonical "whodunnit" fiction or Bond-style adventure fiction, the obstacles and conflict that drive the narrative are not dramatic in my narrow sense (emotionally moving questions of value) but what I called upthread "procedural" - ie "How will Bond get out of this one?" or "How can we work out which of the suspects is the real killer?" In an RPG, answering these questions involves learning about the setting (ie the shared fiction). Typically this learning is the result of (i) the GM narrating more background using whatever the procedures are for that (eg knowledge checks, stating that my PC walks into room X and looks around, etc) and then (ii) acting on the setting (typically via the action resolution rules). The resoution of action declaration contributes to the setting - ie the shared fiction - by adding new information eg now the PCs are on Bytopia, or now this shopkeeper is our friend, or now Demogorgon is dead, or whatever else is the upshot of the action resolution.
Some RPG play takes the above as
the primary, perhaps sole focus of play. That is what I called "setting exporation" play. That play may be exciting - eg finding out whether or not Demogorgon lives or dies should be exciting, otherwise why play it out? - but it won't be dramatic in the sense of involving
emotionally moving conflict. There will be excitement, perhaps anxiety (but probably not genuine fear in the typical RPG experience), perhaps curiosity ("Is Demogorgon vulnerable to the widget we got from Bahamut?"), but not questions of value.
Now I'm not sure about Demogorgon, but at least for Lolth it's not that hard to introduce questions of value. Consider this campaign idea from Underdark (p 23):
Tharizdun breaks free from his chains. He makes his way for the Demonweb Pits. Epic battles unfold as demons aligned with Tharizdun try to block the adventurers' path to the pits. In what seems to be a final titanic encounter, they come upon Tharizdun after he has dealt Lolth a near-fatal blow. As prophesied by the cultists of the Eye, her dying degrades the webbing that holds reality together. With the universe coming apart around them, the Demonweb Pits howling for union with the Abyss, the characters must defeat Tharizdun, allowing Lolth to recover and stitch everything back together.
Then they understand the true meaning of their marks [at the outset of this hpyothesised campaign arc, each of the PCs has been marked with a prophetic, destiny-signifying sigil]. Their destinies were to save Lolth not only by defeating her greatest enemy, but by feeding themselves to her as a revivifying sacrifice. . . . Do they accept oblivion to save the universe, and allow themselves to be heroically devoured? Or do they come up with some other plan so crazy that it just might work, saving themselves along with the universe, and dooming Lolth as well as Tharizdun?
Now, as presented, this is rather railroady, because the PCs have to be marked at time 1 in the campaign, and then get caught up in these Abyssal hijinks at time 2. But there are other ways of bringing PCs into conlict with Lolth and Tharizdun without needing to adopt this particular, railroading strategy.
What I am pointing to in the example is something different: that there the choice whether to allow Lolth to die, or not, involves a dramatic conflict, a confict of value for the PCs that (hopefully) will also be experienced as a dramatic moment by the players: do they save themselves in the short term but at the cost of the world?; do they save the world but at the cost of allowing a newly-rejuvenated Lolth to (i) eat them and (ii) probably conquer that world?; do they find a third way out?
The provision for a third way out is something of a concession to sentimenality. In my previous long-running campaign, it looked like the paladin PC would have to sacrifice himself to save the world and the dead god he worshipped, but then through cunning plans the players and their PCs came up with a way out that saved the world, the god and the PC and destroyed the PCs' biggest nemeses. In these "third way out" scenarios, I think the key to maintaing a dramatic rather than a procedural/exploration tone is to have the third way out emerge and be resolved within the same situation of choice - if the third way out is conceived of, but then implement it turns into a multi-session hunt for just the right McGuffin, etc, then the drama has been replaced by what I have called "setting exploration" play.
Narrative
Like "drama",
narrative has a range of broader and narrower meanings.
In the sense I am using it, solving a crossword puzzle does not involve narrative. Nor does hammering in a nail. I confront the task (blank crossword, a nail that needs hammering), I gird my loins, I get on with the job.
To make it a narrative, you at least need opposition: will I get the nail hammered in before the room floods and we all drown; will I solve the crossword in time to decipher the serial killer's next victim? (The opposition probably doesn't need to be on a timer, but for these sorts of mundane tasks to become narratively engaging that is the easiest way to do it, I think.)
Tomb of Horrors has no opposition in this sense. Like a crossword in the newspaper, or having all afternoon to repair the clothesine, you just keep going til its done or you don't want to, or can't, do it any more.
White Plume Mountain is similar, but less like a crossword and more like a boardgame, because unlike ToH you get to roll lots of dice that will help decide whether you win or lose. But there is no narrative engagement in WPM as written.
There is a contrast, in this respect, even with GDQ, where there is at least a loose sense that if the giants and drow aren't defeated then they will conquer the world. The A-modules make the narrative stronger again: unless the slavers are defeated, they will continue enslaving people. Still, in play that narrative motivation mostly drops out of the picture and it is much like White Plume Mountain.
A module like Reavers of Harkenwold (? the well-regarded one that comes with the Essentials DM Kit) makes the narrative framing a more serious part of the module, as the threat from the raiders serves not just as a hook, but as an ongoing element of the adventure as it is played out.
But it is still not dramatic, in my narrow sense, as written - the PCs are analogous to James Bond but aren't forced to engage with serious conflicts of value.
I don't think I understand enough what you are actually looking for to be able to understand why the Nentir Vale does it for you but other D&D settings don't.
I think I mentioned upthread that I don't use the Nentir Vale. I use the 4e default cosmology, which is outlined in Worlds & Monsters and then again in the DMG, appears in snipptes and sidebars in the PHB (especially races) and the Power books, and is developed in more detail (sometimes detail that I ignore) in MotP, Plane Above, Plane Below, Undedark, Open Grave and Demonomicon.
What is present in that cosmology is a need to make choices that involve conflicts of value. For instance, choosing to wipe out orcs, and therefore (utimately) Gruumsh, means choosing to destroy one of the more powerful allies the gods have to hold off the primordials and thereby stop the destruction of the world. Choosing to oppose drow, and thereby Lolth, has the implications spelled out in the extract from Underdark I posted above.
It's true that the obtaining of these choices involve certain assumptions. They involve an assumption that heroic and paragon play (vs orcs, vs drow) is linked not just in the abstract but in the experience of play to cosmogical matters like the status of Gruumsh and Lolth. I play D&D in that fashion by default (influenced by the original Oriental Adventures), but I think 4e in both its presentation of PC races and of monsters, and in its integration of epic tier into the core of the game, strongly supports that assumption.
The obtaining of these choices also invovles an assumption that the game will involve fighting orcs, drow, goblins, gnolls etc and not (say) ankhegs, kruthiks, bullettes, etc. As I've said in the past, if the 4e party is all hafling rangers who worship Melora and Avandra and fight kruthiks, then the game won't deliver, in and of itself, the sort of dramatic payoff I enjoy. Luckily for me, fighting orcs, drow, gnolls etc rather than giant insects, and playing thematically "meaty" races like dwarves, elves, tieflings, humans of fallnen Nerath etc rather than halflings, is the norm for my group.
Under these assumptions, the default conflicts into which even low-level PCs get framed are laden with the need to make choices that aren't free of dramatic cost, because they involve choosing one vaue over another.
I think there are other ways, too, that 4e makes dramatic conflict easier than many other versions/settings of D&D. For instance, early in my 4e campaign the PCs received a surrender from an enemy mage, who promised to work with them. The paladin PC took her under his wing. Not too much later she was killed on a battlefield imbued with negative energy. The next turn she rose as a wight and attacked the paladin - a paladin of the Raven Queen, who therefore had to defend himself against an undead (whom he's sworn to destory in the name of the Raven Queen) but by killing the mage (or an echo of the mage) whom he had taken under his wing.
There are multiple reasons I think that this sort of thing is easer in 4e than in other versions/setting. Some involve its mechanical features, and so aren't relevant to this thread. But some involve its story elements: because the game builds on core D&D tropes (like the nature of undead, and the hostility between clerics and undead) but gives many of these a deeper significance than they have often had, and link them direclty into the cosmology (eg its not just a paladin vs a wight - in a sense "natural enemies" - but a paladin of the Raven Queen vs a low-level agent/minion of Orcus, hence
sworn enemies), I find it easier to just follow my D&D instincts, and reach into the bag of tools the game gives me, and generate these situations which have not just procedural but dramatic weight.
For me, at least, the extent to which a system and setting facilitates this sort of thing is important, and the 4e cosmology as expressed through the published 4e GM-side story elements (eg monsters listing) is one of the better I have encountered. (As I've mentioned, OA adventures is another: players are hooked into a framework of family loyalty, political loyalty, and honour; and so are their monstrous antagonists, via the Celistial Bureaucracy; so it is similarly easy to reach into the grab bag and pull something out that will have not just procedural but dramatic weight.)
) On and on and on.
