That's why I like the fact that feats in D&D Next mostly apply static modifiers that aren't conditional. When they do add new abilities, they don't add complex ones. Also, you don't get very many feats so you don't have to worry about the complexity growing out of control. The players who find them too complex can just take stat bumps instead. You don't have to worry about too many temporary modifiers from spells because all buff spells require concentration, so only one buff per caster.
That kind of stuff does happen with veteran players. They were the ones I was talking about. That happened nearly every round of combat in our group of friends who'd been playing 3e/3.5e for 4 years straight.
Although my group composition has mostly changed, I don't believe that to be the reason our sessions no longer get bogged down in discussion of modifiers. I believe it's because we're running D&D Next now. There are basically 0 modifiers applied at the table. Turns now go:
"I attack the already damaged Orc. I hit AC 13."
"That hits."
"9 damage."
The point is that there ARE currently no modifiers in the game. Adding one might not be the end of the world. But once you add one then it becomes part of the game design philosophy to have them in the game, that's when you start adding a second, third, and then hundreds.