There is a personal saying of mine that I have used in game a few times: "Respect is given, trust is earned.'
If the player is essentially a decent person, give him a chance to try to earn the respect of his DM and his fellow players. Let him use other people's dice for a while. Then, after a probationary period of a few months, tell him you will let him use his own dice, but you will occassionally look at them. Perhaps he will get to the point where you can fully trust him again.
Personally, I do believe in giving people second chances. I am very far from perfect myself.
Ousting someone is a possible solution, but there can be hurt feelings. I have had one situation where a group split up because of hurt feelings. (In this case, it was a player deciding to betray her fellow players and a few real world issues besides. She was completely oblivious to the hurt feelings of others.) So, do try to consider other people's feelings.
Clearly, the cheater is in the wrong. However, sometimes a person who gets a second chance becomes better than before. I wish you well with this problem, BVB.