Yo, AR,
As a novelist and DM, I feel as though this is a topic close to my heart.
It's similar in a lot of ways. For example, establishing an agreed-upon logic. If magic can do anything in a fantasy story, the reader usually gets dull. Magic is only interesting if it is mysterious, powerful, but ALSO kept in check by limitations that adhere to the world's internal logic. The swordsman should, barring injury or special circumstances, radically change his style for no reason, either for better or worse -- he shouldn't be able to defeat twenty guards early in the book and then get beaten down by three guards of no better ability a chapter later (unless the guards have the element of surprise, the swordsman is exhausted or drugged, or something else changes the balance of power). In a novel, those things break the suspension of disbelief. In an RPG, we have rules to ensure that the rules stay consistent unless the DM rules special circumstances -- for example, a Polymorph curse that can't be dispelled or removed by ordinary means, but must be broken by true love...
Other things, however, are different.
The storyline is usually different -- the "try and fail, try and fail, try one last time and succeed" storyline doesn't work, and usually gets replaced by "defeat little minion, defeat bigger minion, defeat BBEG, sell loot". In addition, characters don't have the smooth progression of most novels, first because of the random nature of what they can do (they don't always succeed or fail right when it is dramatically appropriate for them to do so) and second because the characters are somewhat closer to getting to write their own scene. Many times, this results in characters who are unwilling to fail because they are overly self-aware. This isn't just another DM attacking metagaming -- it's HARD for a player to do something genuinely long-term self-destructive, because they're looking at the situation without the same emotional intensity as the person in the story would have. An author usually has the objectivity (and interest in the plot) necessary to give that emotional intensity to his characters. Because D&D is more about problem-solving, defined by success, most players don't get this -- and the DM is often equally to blame.
I usually think that the DM is in charge of the setting, much of the backstory, and the Initial Problem aspect of the plot, along with Complications that arise as necessary. The players are in charge of most of the important characters. The story is limited to the PCs -- you never get to see a scene from the POV of the villain, which is used in many novels either to make you hate the villain more or to begin to sympathize with him.
More as I think.