First, if you're designing these things yourself, think about using fewer Elites, if any. Your post got me thinking about Elites in encounter and monster design, and I sat down to look at the action economy (among other things) in a blog post
here.
The Gist is this -- an Elite lasts twice as long, but doesn't get twice as many actions (with the exception of an action point, once). So, if you pit your part of 6 against an appropriate group of lvl 4 normals, those normals will get 6 attacks per round. Against Elites, the monsters get 3 attacks per round (assuming only standard actions are attacks).
If the party takes down one normal opponent per round, the group of 6 gets a grand total of 21 actions to attack the part, while the Elites get 12 (+3 maybe for action points to 15).
6 normals: 6 actions 1st round+ 5 second round , +4+3+2+1 = 21.
3 Elites: 3 actions first round+ 3 actions 2nd round+ 2 actions 3rd round +2+1+1=12, then +3 for action points = 15.
Remember that in most cases, whlie Elites built from templates have more options for attacks, they don't actually get more attacks, and their attacks per round are not more effective.
Also, a larger number of opponents makes it more likely that the monsters will be able to take advantage of lapses in tactics from your players -- many more opportunities to flank, to attack someone other than the defenders, etc.
Also, I'd look at throwing in some minions if you're really looking to make them sweat. The party doesn't have a controller -- which means they have limited access to AOE attacks. That means that it's going to take an actual action from a character to take out a minion, meanwhile all of those minions are out there getting in the party's way, providing flanks, causing damage, etc.
You cam dummy up your own minions quickly by taking a typical creature of that level, stripping out all special attacks, changing their damage to something like minimum damage. That's not perfect (I often decide the damage is a little low and add a point or two) but it works, and it's fast.
-j