I think spell casting focii are in the game as they are iconic in fantasy, such as the Wizard waving his wand, or the Cleric brandishing her holy symbol. A spell caster can still use a shield while having a hand free or holding their focus, so the main thing they're losing by having to have a hand free or focus in hand is the chance to make opportunity attacks with a melee weapon or the AC benefit of a shield. Mechanically, the impact is somewhat minor.
Where it causes some issues is the weapon+shield user or two-weapon fighting needing to sheathe a weapon, cast a spell, then draw the weapon next round. At most mechanically, they're losing the chance to make an opportunity attack with a melee weapon every other round, and they're using their free object interaction to sheathe and draw a weapon when they do so. But again, the mechanical impact is somewhat minor.
Ideally, there should be some benefit for spell casters using a focus to encourage that iconic use, if so desired in a particular fantasy setting. For example if they're using a magical focus, it could add between +1 to +3 to spell damage, or allow them to use their spell casting ability modifier on cantrips if they don't already get that benefit. It is a little strange that we have all these magic weapons for the non-spell casters, but limited similar stuff for spell casters (e.g., Rod of the Pact Keeper, and Wand of the War Mage).
The other approach is to disallow sheathing and unsheathing of weapon when spell casting. You could make use of material components the free object interaction for the turn, if they're not already in hand. And I would get rid of the distinction between somatic and material components to simplify the rules, but then the War Caster feat needs an update too of course. The point of this path is to force one of the following options:
- One-handed shield use, no weapon - permitted with spell casting, if you are proficient in shields.
- One-handed weapon use, no shield - permitted with spell casting.
- Two-handed weapon use - permitted with spell casting or not?
- Two-weapon fighting would then be disallowed with spell casting.
- Weapon + shield would also then be disallowed with spell casting.
I don't really see the need to limit spell casting in this manner, particularly as the mechanical benefits of allowing all of them to do spell casting is relatively minor vs. what is already workable by the rules as written now. We should have a system that permits a variety of builds.
If you want to restrict it, perhaps the other option is to restrict it by proficiency. You cannot use a shield well without shield proficiency. Make two-weapon fighting the same: martial characters (Barbarian, Fighter, Monk, Paladin, Ranger, Rogue) get sufficient training to two-weapon fight with light weapons, or weapon+shield if they have shield proficiency; whereas the spell casting focused characters (Cleric, Druid, Sorcerer, Warlock, Wizard) would need to pick up either the Dual Wielder or War Caster feat to do so. But I don't really see any need for such a limitation.