And personally, I see the "I'm going to burn the village down for fun" evil just the same as "Teehee, I'm a kender and I'll play a kender like kender are!" They're both done for the sake of being annoying/disruptive, and alignment doesn't matter when it comes to the player.
This.
The thing I find unacceptable is disruptive behavior. Obnoxiously messing with other PCs is not cool; being the guy who refuses to get on board with the party's goals is not cool; acting sociopathic toward random NPCs is not cool. I don't care what your alignment is.
Yes, there are people who use "I'm eeeevil!" as an excuse for these things, but if you don't allow them to play evil characters, they'll play Chaotic Neutral - or, in 4E, Unaligned - characters and keep right on going. Heck, even if you mandate good alignment, they'll still find ways to do those things. The way to stop them is to a) explain that you want them to stop the disruptive behavior, and b) if they keep being disruptive, kick them out of the game.
If somebody has an idea for an evil character who can get on okay with the rest of the group, who doesn't always have to be coaxed to sign up for the quest
du jour, and who doesn't commit senseless mayhem, that's fine by me. In fact, an evil character can add an interesting dynamic to an otherwise heroic party. Raistlin Majere is a good example of this. He's a ruthless, power-hungry bastard, but he still works well within the Heroes of the Lance, and he challenges the other Heroes to confront their own dark sides.
On the other hand, Tasslehoff Burrfoot is a perfect example of a non-evil but highly disruptive character. His antics can be fun in the books, but in a D&D game they would drive most DMs and players absolutely bats**t. I'd take a Raistlin PC in my gaming group over a Tasslehoff PC any day of the week. In fact, I'd probably take Raistlin over Sturm, who skirts the edges of the Lawful Stupid paladin archetype.
Secondly, they know that if they do stupid things to each other, like stealing or lipping off or whatever other disruptive actions I've seen players make over the years, the other guy is well within character to kill him in his sleep.
This is actually a nice argument for including at least one well-played evil character in every party. Great way to cut down on intra-party squabbling. As I recall, even though the rest of the party was constantly being pilfered and pestered, Raistlin never had much trouble with Tasslehoff...