Damage per Second is a problem with some 4e builds. One way to approach 4e characters is to build up effectiveness per round, while neglecting effectiveness per second.
Many low-damage effects, condition tracking, and the like all increase bookkeeping.
On top of this, the DM has to really be on the ball to keep the game going along. The DM has to move creatures instantly without looking for advantage, has to know every effect like the back of their hand, and has to have a system for keeping track of monster HP/defences/status that doesn't take time -- a slow DM can easily double each player's turn, and then take three times as long themselves for the Monster's turn, as they have to record and add up damage and effects on monsters.
I say "don't look for advantage" as the DM because, if you want to challenge the players you can just use tougher or deadlier monsters.
Having characters that are optimized both for damage per round and damage per second is also key. And having players understand that their goal is to finish their turn in the least amount of time possible...
As an example, you can have everyone use power cards, cut into individual squares. If you have your standard action power card alone, face up, in front of you when your turn happens, give the player a +1 to attack (or, if generous, a +1d4). If you prefer to be cruel, say that if you don't have your standard action power card alone, face up, in front of you when your turn starts you have to use your designated default at-will (unaugmented) power, and reserve the +1d4 die to hit to attacks that occur in the first minute of your turn (use a chess timer or something).
You'll be amazed at how fast combats start to fly.
If your players are relatively low op, and find that challenging combats take too many rounds, halve monster HP and boost non-minion monster counts by 50% (ie, increase the XP budget by 50%) while using only MM3 and MV monsters.
On the player side, you can concentrate on building characters with high DPR and high DPS (damage per round and damage per second). Status effects are fun and effective, but they don't end combat any faster.