Hiya.
I think you're heading in the wrong direction to try and curtail that kind of min/max behaviour. The bottom line of it is that he's doing this to get attention and have others oooh and aaaaahh at how "kewl" he is. He may not even be aware of it himself...but you did mention he is a GM, so maybe he's used to being "the focus of attention". Anyway, to quote "Joshua" (the computer from War Games):
The only winning move is, not to play."
I had to do this two sessions in a row waaaaaay back in the late 80's when I was DM'ing. One player was horrible for 'min/maxing' his guys to the point of minor cheating (call it..."selective rules remembering"). Anyway, the way I got him out of that habit was to simply not play that game. When monsters attacked him, I'd pick up some dice, roll them and then say "He misses", and move on to the next guy...sometimes not really even looking at the dice. When he attacked an orc, before he rolled the die I'd blurt out "You kill him" and immediately move on to the next guy. Eventually he got annoyed and said "You're not even rolling!". I followed that up with a handfull of d20 rolls and said "OK, there. I rolled for the next 7 attacks against you. I'll tell you if you get hit". Needless to say, each of the next 7 attacks were even quicker with a simple "They attack you and miss...Next!". He was doubley perturbed about me not letting him roll his stupendous damage to hit the orc, for example. When his minimum damage is enought to kill the beasty 4 times over, it's pointless.
What was the result? He got bored
very quickly...and not a triffle bit miffed. After the second game I confronted him about me "sensing he wasn't having much fun". I basically pointed out the absurdity of his character, and showed him all the bad guys they fought so he could see just how nigh-pointless it was for us to waste our time watching him roll and kill things. There was zero 'excitment' for us watching the inevitable, so I decided to just forego all that bordom and move on to people who *could* miss or not kill in a single hit. He took it to heart and realized that it was kinda silly.
He made a new guy and since that day has had more or less 'normal' characters who may excell at one particular thing...but are no where *near* the min/max'ed monsters he used to make. He enjoys the story and background of his characters more than anything now.
So...after that long-winded brain purge...I suggest doing something similar. Don't 'bother' rolling or just roll mass dice. Don't let him roll damage, just say "You kill it". Make blanket assumptions about skill checks for him..."Everyone make a Climb check at DC 15...except you. Don't bother, you make it"...when he complains, shoot him a sarcastic/annoyed look and say..."Oooo...ok...fine. Roll..." And when he rolls 12 and adds his +18, say (with sarcastic emphasis) "Didja' make it? ... ...".
In my experience, the only thing that catoring encounters to challenge this kind of player/character is going to lead everyone else in the group to try and min/max themselves in order to feel special and cool too. By hand-waving and otherwise 'assumingly gloss-over' the uber-PC, everyone in the group (sans the affected player) is going to see their characters as interesting and cool and the 'uber-PC' as lame.
If that doesn't work...just outright cheat and have something FAR superior to him kick his ass EVERY SINGLE ENCOUNTER. Either he'll quit in disgust, or start to make 'normal' characters simply because taking 4 days to make a new uber-PC isn't worth it anymore if he dies in the first 3 minutes of play.
^_^
Paul L. Ming