every single group I have ever seen believes or claims to be driven by story...regardless of the truth of that claim. A DM wanting to play to that false expectation can easily be accused of railroading.
This is why I think that the key word in the Forge's "Story Now" is not "Story" but "Now". Every RPG experience will generate a story - it's all about how that generation relates to the purposes and the procedures of play.
I suspect that some of what drives railroading is the difficulty of preparation.
<snip>
Sometimes, I think that "railroading" is merely lack of prep time combined with a DM misinterpreting the party's interests.
<snip>
Which is not to say that railroading isn't real, or doesn't happen for other reasons.
I think the felt need for preparation can be part of it, yes.
I think there's also a connection here to worldbuilding. One thing that I have found can help with more improvisational GMing is to have the worldbuilding in pretty broad-brush terms (the Points of Light presented in Worlds and Monsters, the 4e PHB and MM, and the DMG is fine for me), and then do a lot of spontaneous creation of detail as play unfolds.
This is the sort of thing that in Burning Wheel is mechanically formalised via Circles mechanics (players get to create NPCs) and Wise mechanics (players get to create backstory) but, especially if the group are comfortable together, you don't
need those sorts of mechanics to have just-in-time world creation (whether done mostly by the GM, as in my case, with a few key bits of input from the players, or whether done with the players taking a bigger role, as in BW or the discussion in the 4e DMG2).
Conversely, a GM who has put a lot of effort into worldbuilding from the get go may well have bits of the world that s/he wants to show off in play. If the world also contains a built in thematic or moral orientation that the players have to buy into if they are to engage it at all, then I think railroading can easily ensue.
As you know, I'm a little bit down on 2nd ed AD&D for its railroading tendencies, and that opinion is definitely shaped by some bad experiences, with GMs wanting to dominate the unfolding plot in the interests of telling
their version of the incipient story.
The most prolonged example of this that I endured involved a GM with a well-developed homebrew world, a prophecy, and a PC played by a somewhat irritating player who was (of course) at the centre of the prophecy. From that description alone you can probably see all the hallmarks of impending disaster! What was interesting was that the group was quite big (seven players, I think, or six at least) and as a result we were able to build up our own story dynamics via intra-party roleplay. Which kept the campaign alive for six months or so, but also led to a bit of a battle-of-power between players and GM, which the GM resolved in his favour (for the short term, at least) by time-travelling the whole party 100 years in the future, thereby invalidating all the background that we players had mutually created and that underpinned the intra-party dynamics. I left the game not long after (partly because of the change, partly because I got a job that reduced my free time), and I think it ended shortly after that.
The speed and ease of DM prep in 4e allows you to do this in ways that earlier editions would make nearly impossible and certainly impractical. Which is what, I think, leads to the mostly static dungeon style of adventure for old-schoolers. The DM can prep a horde of adversaries, a host of encounters, sometimes connected by a few conditionals (both explicit choices and simple geography) and then leave it up to the party to "solve" it. Reducing the ability of the party to "go off the map" reduces the chances of the DM getting caught with his pants down
I think the "puzzle solving" dimension you allude to here is also an important element of at least some old-school play (Tomb of Horrors and White Plume Mountain are poster children for it, I think). Gygax talks a bit about this in his concluding section of his PHB, and Lewis Pulsipher in the early days of White Dwarf used to emphasise it very much - the the GM must build the dungeon and then not toy with it (especially not on the basis of narrative considerations) because this will undermine the ability of good playes to get and advantage by using divination, scrying etc.
Whereas in my game the "puzzle" dimension is more about how to leverage the unfolding situation in the direction you (the player) want: so it's more about engaging the mechanics to move the story forward in a certain fashion, then about scoping out the "dungeon" to maximise your prospects of a successful foray.
These are very different playstyles, and they take different approaches towards preparation, but I don't think preparation is at the heart of it. You can run improv/just-in-time D&D using AD&D mechanics (I've done it, especially in Oriental Adventures), though it doesn't support this approach as well as 4e does.
In groups like mine, I think some of their reactions are learned from bad experiences. They are even skittish about things like Action Points (or its cousins), or any other open admission that we are playing a story or that anything like Narrative Causality might be functioning.
4e isn't unique in the way it tries to handle this issue, but it is unique among editions of D&D. The transparency of the system - power mechanics, item mechanics, earning and spending action points, DC and damage ranges, etc - all combine to give the players a confidence that the narrative cauality won't hose them or their PCs. And you don't even have to spell all this out - I think that the confidence in the mechanics to give ever fair points of input, and therefore the requisite degree of control over their PC's fate, emerges out of play. Partly because the players encounter it in their own resource management. Partly because the GM has such clear mechanical support for saying "yes" or "yes, but". And partly because the resolution mechanics tend to ensure that no single choice or decision by player, or single adjudication by the GM as to who the monster attacks or where it moves, bring everything crashing down. So the play iteslf is transparent and forgiving in this particular way.
AD&D's lack of this sort of transparency is what I think makes it less reliable as a system for this sort of play, but I'd be interested to hear how you (or others) worked around its deficiencies.
I think D&Dnext could achieve something similar to 4e, in this respect at least, if bounded accuracy works out as I hope it will, but at the moment I am a bit uncertain - and the "rulings, not rules" rhetoric fuels my uncertainty, because to me it focuses too much on loosening things just where they need to be tight if players are to confidently take risks (eg DC setting, damage values, etc) rather than getting the GM's adjudication where is can be put to best use (framing challenges, determining narrative consequences, elaborating complications, etc).
1) The DM is not particularly good at impromptu content creation or has poor improvisation skills. Railroading is a "safe", risk averse style for such a DM. There is no deep water, so to speak.
2) The DM gains his fine by authoring/actualizing a story. Without such editorial control they wouldn't be in the chair. He gives a tacit nod to mechanical influence and creates the illusion of PC agency but in truth its just a dog and pony show. Further, oftentimes, the players will be willing co-conspiriators to this DM Force (wink, wink, nudge, nudge, say no more, say no more) as they understand that without the DM's interestes sated (story-time), either they may be forced to assume the mantle or the group might be a ship without a sail.
3) The players actually want a railroad. They do not want to make decisions and/or are paralyzed by parsing the content at the table and the subsequent pro-activity required to move the plot themselves. They effectively want to roll dice, listen to a story composed by a DM and perhaps imagine their characters (or attempt to immerse within) participating.
[MENTION=386]LostSoul[/MENTION] has posted sometime in the past few months (I can't remember which thread) about your number 3 - players who don't want to make choices, and who prefer to "immerse" in the story that is being created by the GM in which their PCs figure as "protagonists", in a certain sense at least.
I also think that there is a variant on (3), where the player has been "educated" by other RPGers into thinking that the sort of passive acquiesence you describe is what it takes to be a true RPGer - play your guy, emote your lines, but don't shake up the big picture.
In this approach, as in your (2), the tacit nod to mechanical influence may play a role, but it is clearly subordinated to the GM's control over plot: for example, if the PCs are losing a fight the GM won't necessarily fudge, but will introduce some NPC or comparable device to make sure that the direction of things is not derailed.
One thing I find interesting is to see where other RPGers draw signficant boundaries, and how that often differes from my own boundary-drawing tendencies. For instance, the first time I mentioned how I handled the one (and to date only) TPK in my 4e game - namely, the PCs regain consciousness in the goblin prison (except for the one whose player wanted a new PC, and for the one who was really dead - that one literally came back to life at the will of the Raven Queen) - some posters here talked about railroading, or invalidating player choices, and invoked the spectre of deus ex machina.
Whereas it seemed to me (and still seems to me) that my way of handling the TPK - given that it had already been established that the goblins liked to take prisoners - preserved the narrative significance of the loss in combat, while also preserving the continuity in the campaign that was desired by all participants.
In other words, for me what is significant is not the similarity between the deus ex machina and my own approach - PCs continue to live, and thus the campaign goes on - but the similarity between just letting the TPK stand and my own approach - the direction of the narrative changes unexpectedly in response to (i) choices made by the players, and (ii) the consequences (in this case, failure) that resulted from those choices.
The speed and ease of DM prep in 4e allows you to do this in ways that earlier editions would make nearly impossible and certainly impractical. Which is what, I think, leads to the mostly static dungeon style of adventure for old-schoolers. The DM can prep a horde of adversaries, a host of encounters, sometimes connected by a few conditionals (both explicit choices and simple geography) and then leave it up to the party to "solve" it. Reducing the ability of the party to "go off the map" reduces the chances of the DM getting caught with his pants down (a significantly bigger crisis in <4e editions.)
(Just as I had finally succeeded in attracting the attention of an order of Paladins and creating the hero who should've gone on to defeat me, more's the pity.)
) I keep running into games (usually Indie, anymore) that look great, but somehow don't work out as well in practice. I have high hopes for the Cortex+ System from the Marvel rpg. I'm hoping for a solid fantasy version to bubble up. 
