The systems most effective at enhancing or even inspiring role-playing generally have something to do with mental condition. At least in my experience. The Unknown Armies meter system is pretty much unparalleled in my opinion. It gives a real sense of consequences that I find lacking in other systems.
CoC's sanity system is pretty good but until you actually inflict a quantified condition of insanity on a character there are no real repercussions to losing sanity points. Characters with lower sanity are closer to slipping over the edge but don't really suffer from any sort of condition that would direct role-playing, IIRC. In the original Chaosium rules your POW(er) was usually the main resistance to madness and it didn't suffer any loss as your sanity dwindled. I think in d20 it's a Will save. The system does provide a meter for players so inclined to role-play their descent but it really requires the players initiative.
I was never really too fond of the Path system in the Storyteller series. It was mainly in Vampire but there was a humanity variant for Mortals that appeared in the games that featured those character types. I just never really got it. Mechanically it made sense but it seemed very contrived to me. I did like the Demeanor/Nature system though. Demeanor was just a loose guideline but Nature actually enabled you to regain precious Willpower by acting in accordance to your nature, oddly enough.
I've been running d20 Conan and the system they have in place for Codes of Honor is pretty good. Having a code of honor gives you a flat +3 bonus to all Will saves and an additional +3 against Corruption (the slow erosion of your souls and sanity). The Codes themselves are defined as Barbaric, Civilized, Chivalrous, and Mercenary. I believe there may be others in supplements I don't yet own. Each one has very specific rules as to what you can and can not do and still maintain your code. They are easy lose if you don't act in accordance to their directives, nigh impossible to regain, and very well suited to the genre. You can also have No Honor if you lose your initial Code. That requires a feat and mechanically acts similar but with reduced benefits. It also puts a taint on you that some sorcerers and outsiders can sense.
Alignment as a role-playing tool has always been pretty poor. I see its intention but I think most gamers view it as a straight-jacket instead of a quality. I've largely rid my games of alignment except in certain cases. I do use it to define a character's essence but rarely have the need to use it as a tool. If a character is an outsider they retain their alignment quality regardless of there actions. This actually led to an interesting character arc for one of my players who wanted to play an incubus that reformed over time. He had to go through a whole lot of hardship and scourging to lose his evil quality. Eventually he actually became human and died free of the evil that had damned him.
Lately I've taken to offering role-playing experience awards in a very codified manner with guidelines from
this system By Clinton R. Nixon. I had to polish the edges to my liking but it is really effective. Although it didn't help the worst power-gamer in my group. But that is another issue.
