Yeah, I think Sejs has the right idea.
I've just DM'd two groups recently, the first of which disbanded because of OOC conflicts between the characters, and the fact that one of the players was being a REAL

, and a couple others were just being asses. I'm actually thankful to this first group... I'd never had such a sucky bunch of players before, and it taught me a few things I had to do for the next game.
In the first game, the

was the worst rules lawyer ever. He would contradict me on simple little rules, halt the game for five to ten minutes looking it up, and the worst bit, is that the PURPOSE to his rules lawyering was so that he could have his character screw over on of the other characters, since he had a grudge against that character's player.
He would look over the DM screen, see my dice rolls, and when I wanted to fudge a roll because it would make something cool happen, he'd call me on it, in front of the other players.
So the next game, in order to head off these kinds of game breaking things, I declared up front, a few things:
1) I don't always know all the rules off the top of my head. If you think I have a rule wrong, say so, and I reserve the right to follow what you say. I also reserve the right to ignore that rule in the name of SPEEDING UP GAME PLAY. When we have TIME, we can go back and look up the rule. If I was wrong, we will play things correctly from then on UNLESS the RAW aren't as much fun, in which case I reserve the right to ignore the rule and use my own. I will be consistant about this.
2) I sometimes fudge dice rolls. If you catch me doing this, SAY NOTHING. Ignore it. I -will- try to be fair, and honestly, I fudge dice rolls in your favor about as often as against it.
I also threw in a bunch of rules about player-to-player conduct. I do NOT want to see a group break up over such pettiness again. Thankfully, my new group is darn cool, and we're having a BLAST.
But this whole rules issue has indeed come up a few times. But because I've declared to everyone my stance on rule challenges, everyone is happy with the way I do things. I make a quick judgement call, move the game along, and then when we have some down time or the spotlight is temporarily on someone else, they look up the rule and show it to me. Things have been fast, fun, and things are getting done. Sometimes I've gotten to eat my words, but the thing is, it worked. We got through the conflict. Next time, we'll get through it better, but both times we had fun, and that's what makes all the difference.
If we WEREN'T having fun, there'd be a bigger problem. The previous group, with the rules lawyer (and the other OOC garbage) made for no fun at all. It was horrible. I really like the way we're handling things now.
But I now think it's very important that you talk to your players/DM about this issue. How do you handle rules disputes? You need to agree WITH YOUR GROUP how to handle such things. My group is happy with my way, because it's fast. Another group might really want to be sticklers for the rules, and they might enjoy that. But you won't know until you ask. So ask! Talk about it! That's the most important thing, I think.