Going back to something constructive:
What if specialization meant the Fighter got options in addition to a standard repertoire of options. These options would not be more powerful, but they would give him more versatility.
For example:
Weapon Specialization (Heavy Blade) - Whirling Blade
After you made a succesful melee attack, expend one Fighter Surge to make an attack against two additional enemies you have not attacked yet this turn.
(Now, some may argue that making 2 additional attacks is better than 1, but spreading the damage around is usually not as good as an idea as focusing on one target)
Weapon Specialization (Heavy Blade) - Duelist's Feint
When you have disadvantage on attack, you can choose to deal no damage but still make the attacks. If at least one would normally indicate a hit, the enemy has disadvantage on his next attack. If both would hit, you and your allies have advantage against the enemy until the end of your next action.
Weapon Specialization (Light Blade) - Precision Cut
When you have Advantage on a melee attack, you can forego the second attack to deal +5 damage if the attack hits.
Weapon Specialization (Light Blade) - Bleeding Edge
When you score a critical hit, you can deal normal damage instead of critical damage and inflict 3 ongoing damage to the target. The target can make a Constitution Save (DC 10 + your strength or dexterity modifier) to end the bleeding.
Weapon Specialization (Axes) - Brutal Strike
After you make a successful melee attack, you can expend one Fighter Surge to deal +1d12+STR extra damage.
(Instead of making more attacks that can miss, hit or crit, you deal guaranteed extra damage.)
Weapon Specialization (Flail) - Sweeping Blow
When you score a critical hit, instead of dealing critical damage, you can knock the enemy prone. The target suffers disadvantage until your next action and grants you advantage until your next action.
Weapon Specialization (Hammer) - Stunning Blow
When you hit an enemy with a melee attack, you can expend one Fighter Surge. The enemy cannot act on his next action and grants advantage until your next action.
Weapon Specialization (Pick) -
Armor-Piercing Strike
When you have advantage, you can forego the second attack roll and instead add +3 to your attack roll. You can only use this ability against a creature that wears medium or heavy armor or has scales.
Weapon Specialization (Bow) - Rapid Shot
After a succesful hit with a bow, spend a fighter surge - make up to two additional attacks, each against a different enemy close to the initial target.
Weapon Specialization (Bow) - Pinning SHot
When you score a critical hit, you can forego dealing critical damage and instead pin the target. The target's speed is reduced by 30 ft. If it does not free itself but moves, it takes 1d6+DEX damage. It can free itself with an action.
Basically, each of these specialization require you to sacrifice something to gain the benefit and are "relatively" neutral in terms of effect. A bigger list of abilities for each weapon and someone specialized would have a lot of interesting options - but he'd still be just as effective with a non-specialized weapon.
Just with the current rules and without introducing additional systems, I think I've identified these possible "triggers" for specializations
- Having Advantages - Lose the benefit from Advantage
- Making a Critical Hit - Lose the damage bonus from the critical hit
- Expend a Surge - to do something other than the regular action
- Having Disadvantage - Forego the change to deal damage
There may be more, but 4 looks pretty neat to me - now you "just" have to find 4 suitable for each weapon or weapon group. (I went with weapon group). You may have overlapping abilities, of course.