I voted 3, 1 GM + 2 players, because that is the current size of our group.  It has been 3 for a couple of years and probably will be indefinitely.  We've been playing a lot of Savage Worlds which works well for 2 heroes with a few extras.  That game could accommodate any number of players, though.  
I would love to recapture our 3 other lapsed players that are still (somewhat) local, but I really don't think it will happen.  One guy says he would come if I resumed my 3.0 game, but 3 players feels a little light for D&D to me.  I would prefer 4 and have a great (IMHO) D&D game in mind for 5.  If I had 6, I've got a Rifts game that I would love to run.  
So, system matters--what we play.  But more important is who I play with.  We're all long-time friends.  The 2 guys I play with currently are friends of 10 & 20 years.  The others are similar.  I haven't wanted to find a new group or players, although I considered it at certain points.  More likely, I would get into something like 40K or just stop playing altogether.  
So, I am really glad to have a small group of really good friends to game with.