Let me see:
1. Pre-High School (1976 - 1982) - just with my family and a few close friends. ~6
2. High School (1982-1985) - As President of the Math, Computer Science, and Chess club, I expanded the club adding D&D and boardgames. ~20
3. Technical School and early college (1985 - 1988) - two steady groups and I was involved with the Parkside Area Wargamers and ran several D&D games at GenCon when it was still at UW Parkside and a few after the move to Milwaukee. ~100
4. Army Reserve (1988 - 1989), during training as a 91G and my year as an active reservist I had a group with rolling membership due to military moves. ~30
5. Active Navy (1989 - 1990), A school and Power School, steady group alternated DMing. ~10
6. Active Navy (1990 - 1993), group with rolling membership. ~25
7. Medically Retired (1993 - 1997), wife in Air Force, group with rolling membership. ~20
8. Divorce (1997 - 1999), failed efforts to build game club on campus at UAA. ~20
9. Now (1999+), happily married to a gamer, small steady group with a bit of turnover. ~8
The above should be within +/-10%.