I once saw a campaign end and a DM quit because of a nurture/nature debate and the fact that he believed genocide and literal baby-killing (of Orcs, in the FR, long after we had specific, named examples of non-Evil Orcs in said setting, though that didn't come up until he tried to re-litigate this a year or two later) was not only morally justifiable but morally imperative if you were Lawful Good.
I don't think you need to have alignments for this to be a problem. Basically any time you have an exploration of good and evil, and you have an utterly incoherent definition of good and evil, you are going to run into this sort of thing whether you have mechanical repercussions or not. Some groups avoid it by discouraging there from being any repercussions to player actions, whether divine, legal, or social, but then if actions don't have repercussions it's not really a serious exploration of anything but a dungeon.
I'm a bit agnostic on the whole orc baby killing thing, simply because orcs aren't real. I don't even have orcs in my game. The goblins I do have I'm pretty clear on the fact that they have personhood equal to humanity (although there is debate about that, my secret word from on high here would be they are wrong, and the big hint of that is that goblins are a PC race), and there are certainly named non-evil goblins and my hand was forced I'd assess attempts at goblin genocide to be evil.
But on the other hand, gnolls are emphatically not people and there are no non-evil examples. Gnoll pup killing has never come up, but attempts at gnoll genocide would be justifiable. They are all basically evil flesh puppets with an evil creator. But I'd have no particular problems one way or the other with a DM saying for the purposes of his campaign, "All intelligent humanoids have free will and are people.", or conversely designating goblins or orcs as non-people as long as the setting is coherent. And I really wouldn't see a lot in that choice beyond personal preference.
But yeah, once it's been demonstrated that at least one is non-evil, then they are people - you made your choice. You can't have it both ways.
(How this debate would play out in a science fiction setting is somewhat different, and involves questions like, "Are the Alien xenomorphs people, and if they are does that even matter?" And it would be perfectly fine to import that sci-fi perspective to a superficially fantasy setting.)
Not even one of the players was having it though. Especially not the Paladin, who was told he would lose his Paladin-hood by failing to kill cowering, weeping orc toddlers because it was required for him to remain LG...
Which of course is what an actual LG person would say - darn the consequences, I'm standing by my honor. Sounds like the player is qualified to play the character he was running.
Chaotic Neutral has also produced more real problems in my experience, at table, than Chaotic Evil.
The big problem I usually see with CN is people want to play it as CE but without repercussions. Thus after a typically indulgent spree of violence, murder, and theft, they typically are upset with me when I change their alignment from CN to CE in my own records. I've come up with the solution to this though. I bribe them. Instead of just telling them that they've been playing CE instead of CN, when they do something particularly loathsome and disrespectful of the personhood of others, I'll say something like, "Ok, I'll give you 100XP if you change your alignment to Chaotic Evil." And typically the player's underlying motivation is actually "win more", and so typically they take this offer as a good deal. It solves the problem without getting into an argument whose actual root is usually they don't want to be "punished" because it interferes with their "winning".
