I think of Saruman as the potential counter-example here - a game in which players have the choice of how their PC's respond to adversity, and whether they make trade-offs that involve taking power from dark sources (like Sauron) is to me a potentially interesting one. Saruman's motives were self-serving, but not entirely so.
Alignment rules are the main obstacle to this sort of game in D&D, because once the character becomes Evil the game leaves no room for the belief by the individidual ingame that s/he is nevertheless acting in the right (or at least a reasonable way), and it also tells the player that his/her PC has crossed the line into wrong and unreasonable.
Alignment does remove the player's (and, to a lesser extent, the character's) doubts. But I don't think that inhibits the ability of PC's to make power with dark sources and still have moral questions about it.
The question isn't "Is what I'm doing evil," though. The question is "Am I evil?"
The alignment system is an agent of heroic adventure gameplay, so it wants people to choose sides in the cosmic war of ideas, and it tries to make sure those sides are clearly defined. But as a consequence of that, it allows for "evil heroes," and it allows for your goodness to be a cost of doing the right (or necessary) thing. Evil isn't any worse than Good, from a gameplay standpoint. They both have their interesting options. They are both tools to achieve a given end.
With alignment, it should be fairly clear when an action is evil. But it is far less clear when a given *character* shifts alignment. It is left up to the DM to determine, with the advice that alignment is a general description of the character, so it should be broadly applicable, and that exceptions are certainly allowed without an alignment shift. This is alignment's grey area, where the question is "How badly do I want to be Good?"
This meshes better with a heroic game because the world is fairly black-and-white. The moral question should be between those options, rather than about what those options entail. Alignment allows the choice between those options to be a challenging one, because it doesn't penalize you for being anything. The shades of grey happen when characters debate about which one they should choose in a given situation.
This is directly relevant to the gameplay, because it influences their immediate tactics. A character who questions the nature of what she's doing (Is this action Evil?) won't truly be affected outside of interesting character dialogue (doing evil is left a wide-open question). A character who is able to choose the nature of what she's doing (Am I willing to do Evil to accomplish this?) has a direct and immediate clarity of action. Yes, I want to make this pact, no I do not want to make this pact. You know it's evil, but you also know the world rewards evil, too, and that by making this one choice, you won't be evil (though you are aware you may have started on a slippery slope and that the pact will come back to haunt you). It's just a little Evil. What's it going to hurt?
So, yes, Alignment does remove from the table the question of the nature of your action. But no, Alignment does not inhbit a moral grey area. It just shifts it into a heroic framework, rather than one resembling real-world questions of "What is Good?" This is entirely suitable to D&D, which is a heroic adventure game, after all.
That's pretty verbose, and it might be a bit hard to grok, but defining Evil and Good (and Law and Chaos) does not mean that the choice between them is at all clear. Toying with Evil for the purposes of Good is entirely acceptable, and your price is basically your Good alignment (which is a sort of "price of innoence;" you can never be pure, but you will make it so others can be). It does mean that it directly influences your immediate character behavior based on your characters' (and your) choice of what to be. A system without alignment doesn't have such a potent, binding effect on the game. And having Alignment in the game doesn't impede various morally grey characters, it just places them in a heroic context where their choices have certainly made them choose sides in the battle (even if it's 'somewhere between Good and Evil').
My issue here isn't absolutism vs relativism, it's simulationism vs gamism & narrativism - ie it's a game design issue, not a meta-ethics issue.
Right, my intent was to say that removing alignment makes for a "relativistic game," where what is good and what is evil depends on the individual group/campaign/etc. rather than on a coded set of universal rules for the game (Good is petting puppies, Evil is drowning them). So the moral quandries would be relative to the group, not absolute to the game.
Perhaps that wasn't the best choice of words, though.
Mechanical alignment gets in the way of gamist play (because it unexpectedly, and from the point of view of the player's own priorities it pointlessly, leaps up and nerfs the Paladin or Cleric or whomever from time to time) and it gets in the way of narrativist play, in which the players (including the GM as one of the players) want to answer the questions - including, perhaps, ones about the truth or falsity of relativism - in the course of their own play.
Indeed, it does get in the way of both of those: gamism because it forces the hand of "Role Playing" in where it might not be welcome, and narrativism because it alraedy defines a narrative.
Alignment as a core rule in D&D doesn't allow for differing interpretations because it's there to enforce D&D's own narrative: Good Vs. Evil. That's the model of heroic gameplay. That doesn't remove muddy questions as much as it frames them in a heroic setting.
That might not be desirable for everybody, and, indeed, Alignment is one of those rules that should be able to be dropped without much consequence (as I believe 4e will support). But to loose Alignment is, partially, to dilute the power of D&D as it's own game, and to support it more as a toolkit. Alignment is Heroic Fantasy, and D&D wants to be Heroic Fantasy. Without alignment, D&D won't be as Heroic, which is great if all you want to do is kill goblins or if you have your own ideas about exploring the nature of heroism, but it sucks if you want to play D&D as D&D -- as the game where you save the town from the rampaging dragon for loot and power and fame and then do it again next week. That isn't what everyone wants from their game, but it absolutely should be what is supported in the core rules, IMO.
An example - some social theorists and political philosophers believe in the notion of "democratic peace" - that democracies don't wage aggressive war. Players interested in this idea could well play a game in which nation A invades nation B. Part of what would be on the table, then, is whether nation A is really a democracy or not (was its invasion defensive, for example, rather than aggressive?). This could make for an interesting modern-day or super-hero game, with espionage, commando, political/social roleplay, etc. Having rules in-game (perhaps a bit like the old Traveller law levels) which gave a mechanical answer to the question of whether or not nation A was a democracy wouldn't help that game, they'd hinder it.
If you transpose this idea to alignment, you get "If Nation A acted like a democracy before this, then they are still a democracy, but if they contine to wage these agressive wars, we don't know how long they will still be a democracy."
Which instantly informs your actions. Want to protect Democracy? DOVE! Think that perhaps Democracy is overrated if it can't be aggressive? HAWK! Perhaps the Hawk learns that what is replacing his Democracy is something he doesn't want, either. Perhaps the Dove realizes that the line between aggressive and defensive is hazy at best. Perhaps they both learn to sit somewhere in the middle, perhaps they go at each other like mad.
Without that sort of 'political alignment' mechanic, what do we have? Nobody really knows what a democracy is, and the question is moot because OMG BOMBS!
Which is realistic, but rather unsastisfying, just like real life.
Now, that's appropriate for certain games, but if the designers want the game to be about, say, the War on War, that kind of certitude is advantageous. And the designers, I believe, want D&D to be a game about Good Vs. Evil, so describing them and defining them and having roles for them is advantageous.
Dropping alignment has the effects on the distribution of narrative control we have just been talking about.
D&D will still have Good and Evil (at least) and will still define it. I'm all for the broadening of the unaligned category, making alignment effectively something you can switch on, but it will still be part of the core game, because the game isn't trying to make a vessel for all your narratives, it's trying to be Heroic Adventure, and that means, in part, Good Vs. Evil.
I don't think it's silly as a literary, metaphorical device (again, it's not Graham Greene, but what RPG writing is?). It is a gameplay one.
Because it's depiction of evil as self-destructive is something you feel should be left open for each DM?
But D&D has NEVER left that question open to DMs. I'm pretty sure 4e's Evil will still be defined and will still include a self-destructive angle, I just think that 4e's Evil will be more extreme than 3e's...it'll be something you have to choose to be, rather than something the DM tells you you have become. Which is fine for me.
D&D has chosen to be a heroic game where Good and Evil fight with each other. It's good that they're opening it up, but if D&D abandoned it all together, it would be going in quite a different direction than any edition has, more of a narrative toolkit and less of an 'implied setting.' And you'd think they'd be dedicated to that goal, and abandoning other things that tie you narratively to the rules (tieflings come from a lost empire! shaping a spell is taught by the Golden Wyverns!, etc.), not just alignment.