Action surge is a 2 level dip. That delay doesn't make a huge difference in white room, level 20, theorycrafting, but in actual play pushes back whatever your main class's features are by a ton, along with ASIs.
Fighting styles aren't breaking any games. The +2 damage/.5 average damage...