You must have been luckyI have DMed for more than 20 years, and this is the first time I have an evil character on the players' group (running lost mine of Phandelver - 5E).

It depends on the group. I've had evil (and EVIL) PCs in games before, and usually it's more the player that's an issue than the character. Stealing from other party members, threatening to murder, and rallying minions is kinda par for the course for evil, and they don't have to be an issue if you have a mature group who can accept it and play it out in character. Disrupting play by stealing the spotlight (by taking actions while not part of the combat) and intimidating other players is generally not cool.The player is experienced, and his character is a female rogue. He often tries to take treasures they find for himself, has been asking how much damage he would do if he attacked a sleeping character he does not like, and wanted to take over leadership of a bandit group they defeated. During most of the battles he avoids staying near the main action, and often explore other areas while their colleagues are fighting.
So how do you guys deal with this kind of behavior?
Talk to the other players individually and privately. If it's not a problem for them, then you shouldn't interfere. If anyone has an issue with it, then talk to the player about the issues and how to resolve them without hard feelings. If the players cannot come to an accord with you as mediator, then you need to oust someone. Judging by your post, I'm guessing it would be the problem player.I mean, if the other members of the party are not punishing the guy (probably because they have a lot of respect for the player), should I do anything?
Sometimes I find it refreshing to have this character around, but other times it feels like the other players are not having fun as they would otherwise.