Honestly, I ran half of the first floor of the Keep last night with 7 players WITHOUT much scaling up at all and each encounter still lasted a good amount of time. Granted they were 1st level characters going into 2nd level encounters, even the first one with the few goblins took them a little...