One thing I forgot to mention in my OP was that part of my reasoning came from thinking back on how previous editions worked.
If you think about it, 1e-3e basically had an at-will + daily structure. Wizards had nothing BUT dailies whereas fighters and thieves had nothing BUT at-wills. Clerics, paladins, rangers, etc. had a mix of at-wills and dailies. I can't, off the top of my head, think of any class that had an encounter power.
One of my other reasons was that doing something 'special' every encounter makes it all the less 'special'. I find I get bored of my encounter powers solely on the basis that they become standard tactics for me that provide no real excitement beyond what a well considered at-will would provide.
I've always been of the opinion, as a DM, that a rough balance of encounters is along the lines of 2 easy (-2 party level), 2 average (equal party level), and 1 hard (+2 party level), with 1 very hard (+4 party level) every ten or so encounters (in no particular order). For the easy and average encounters which provide the bulk of the player experience, combat should be short and sweet. Four rounds max for easy, six rounds max for average. Only with the hard and very hard encounters would I expect players to dish out the big guns and even then, by doing so, the combat shouldn't be more than about six rounds.
Having twenty encounter powers just starts to become redundant at this point. Not only that, but the amount of times I've seen people suffer decision paralysis, because of encounter powers, would add up to several hours worth of lost gameplay at this point.