Perhaps it's because I'm considering a broader context that includes my own circumstances. I live right near so many interesting things and people that I have an excess of choices of how to spend my free time. I also work as a contractor and set my own hours, so if there's nothing worth doing I can and probably should make some money, or be doing something productive towards moving my career forward. I like rpging enough to belong to message boards about it, but for me to want to devote a large part of a day to playing a game, it needs to be satisfying on a lot of levels, not just an excuse to spend time with people.
If someone asked me to play a game of 4e, I'd probably suggest playing something else, or watching some TV or movie of mutual interest, but then if that failed, I'd just walk to the National Mall or drive to some estuarine beach or go hiking at Great Falls, with or without them. Then again, I don't have any friends that would ask me that.
What puzzles me is how people with families and other obligations aren't as discriminating.