I play with people mature enough to handle being out of the action for a little while without being bored or frustrated or disengaged from the game.
Congratulations.
When I was running an open-access 3.5 game at my D&D Meetup I definitely had this problem. I remember that the foolishness of one player caused a disaster - the PC was a Barbarian and the very selfish player wanted to keep his Rage up, so after one encounter he went blundering on and ran into the BBEG Cleric of the module ('Rahasia'). The other PCs then followed; ran into the chamber, and per the module the BBEG's pet Bone Golem then burst out of its hiding place, within a 5' step of the 1st level Wizard PC. 4 sword attacks doing 1d8 each, some hot dice, and the 1st level Wizard was killed instantly.
The ex-Wizard player could have made up a new PC to resume play once the combat was over. But it was a lengthy combat, with PCs scattered around, some running into more monsters, a general SNAFU. The player got frustrated, kept complaining and demanding attention while I was trying to run the complex battle, I got curt with him, he got nasty with me, and eventually I had to tell him to go away. So the whole thing went very badly.
I learnt a few things from that, notably about managing expectations. In retrospect I think I could have left the battle for a couple of minutes to tell him to get to work generating his new PC, that I would bring him in ASAP, and possibly mollified him. Or not - considering the extreme fragility of 1st level 3e PCs combined with the entitlement mentality the game seemed to engender in some players, just killing his PC like that might have been what set him off.
Edit: Later on I booted the Barbarian player too, and eventually a third jerkish player. Not the highlight of my GMing career. But that was all in 2008, well over a year ago now and I've had no trouble since. In particular with running 4e I've had no trouble at all. Compared to 3e, 4e does a great job of setting and managing player expectations - PCs are more durable, and players are not so inculcated in an entitlement mentality. Plus I learned to be a bit more proactive and discriminating in player recruitment and I have a great group now.