Agreed. It would probably rarely come up. Even if you specifically built a caster around this, I don't think there would be enough V only spells to be effective.
Agreed. Someone earlier proposed building a sorcerer who used Subtle Spell to get around that limit, but that's costing you a sorcery point for every single somatic spell you cast, as well as preventing you from using any other metamagic. Plus you have disadvantage on Dex saves, which is not great when you have a caster's hit points; and the best armors will cripple your speed unless you meet a very high Strength requirement.
All in all, I can't see building a caster who routinely wears heavy armor even if this change were in place. There are just too many penalties.