Libramarian
Adventurer
I agree with all of this, including the part about how in adventure paths the plot is usually just a backdrop and not actually that important in many games. I think a lot of D&Ders nowadays could be called "railroaders by habit" -- they think a good game requires the propulsive force of a tight plot, when really most players are pretty indifferent to the quality of the larger story and just want to do some cool, adventurous things and level up their character. I think any DM who has ever used a flowchart in their prep (my indicator of excessive attention to plot) should try running a sandbox as an experiment to see whether this layer of prep really matters that much. IME the "feeling there"ness of a sandbox game makes up for the looser plot.My own view on this is that narrativist play is not all that popular among many RPGers.
I take it for granted that most players aren't very interested in a railroad game unless the railroaded plot is very much a backdrop for the real action of play (this is my sense of how the typical adventure path plays out).
But it also seems to me - and personally this is a bit more surprising - that many players don't like a game in which non-railroaded story is front and centre. Getting story front-and-centre requires the players to deliberately build their PCs to be thematically interesting, and requires the GM to deliberately frame scenes/situations in such a way as to push on those thematic pressure points, thereby triggering the emergence of story. You won't get this without fairly self-conscious metagaming by the players at the PC-build stage, and by the GM at the encounter design stage, and that sort of metagaming seems to be disliked by many RPGers.
My best understanding of Ron Edwards view of the situation is that RPGers scepticism about story arose from bad experiences with railroading and metaplot, and that once games were designed that reliably delivered story without railroading, via the techniques I've described, then those RPGs, and perhaps RPGs more generally, would grow in popularity. And my own tentative view, informed in part by the backlash against 4e, is that Edwards was wrong - at least among the existing RPGing base, there is a very strong hostility not just to railroading (which is entirely warranted), but to the metagame-heavy techniques that are the only known way (to date, ast least) for an RPG to reliably deliver story without railroad.
The Forge used to talk about "simulationism by habit", but to me it seems more like "simulationism by very strong desire".
To tie this back to dungeon crawls: one thing a dungeon crawl can achieve is some of the same results as "story now" play - a degree of pacing, some sort of narrative escalation in the stakes, etc - without the need for overt metagaming in encounter design. Instead the metagame techniques are concealed behind the ingame contrivance of "the dungeon".
Many of the most highly regarded city settings are basically dungeoncrawls: you have a big map keyed with encounters and the players wander around triggering them. And just like the best dungeons, as simple and basic as they are to run, they're actually quite sophisticated in terms of where and how they disregard simulationism in favor of a higher density of fun stuff.