I rarely see this issue raised with dual wielders or two-handed weapon wielders, almost always, it's "sword and board" (usually making the argument that a shield stays strapped to your arm even if you open your hand to use it while spellcasting).
I would simply apply a dose of common sense to the rules as written:
Getting Into and Out of Armor (p146, 2014 PHB):
The time it takes to don or doff armor depends on the armor's category.
Don: This is the time it takes to put on armor. You benefit from the armor's AC only if you take the full time to don the suit of armor.
Doff: This is the time it takes to take off armor. If you have help, reduce this time by half.
Shield: 1 action to Don, 1 action to Doff.
I would further look at the 2024 Player's Handbook, p. 219 where the rules state "a creature can... wield only one Shield at a time."
(Note: Don and doff times are omitted for shields in the 2024 PHB)
In my head, the choice of "wield" (instead of "wear") is telling - wielding does not mean "carrying" - it means "to
hold and use" (that's what the Oxford English Dictionary tells me, emphasis mine).
Therefore, one does not gain an AC bonus for simply "carrying a shield strapped to one's arm" - one must actively dedicate a hand to "hold and use" the shield.
Since a shield is strapped to your arm, it is not "dropped to the ground" when not in use (just as a weapon strapped to one's wrist would not drop to the ground) but the game already assumes that all items in your inventory are considered "carried" unless explicitly dropped (except for items like armor which are worn/donned to gain their benefits, but those have their own explicit don/doff rules). Similarly, the game assumes you cannot use a carried item unless it is "Readied" - usually one's carried weapon (and shield) are readied in advance of combat, so they usually start in a "Ready" state.
Now we can apply a couple of common-sense rules for "hand economy" (assuming here the typical humanoid "two hands").
1. A character has two hands with which to wield/hold items.
2. The rules for some weapons explicitly require both hands be wielding the weapon in order to attack with it. Likewise, some items do not grant a bonus for wielding two items of the same type ("a creature can... wield only one Shield at a time").
3. Anything that does not explicitly require two hands requires one hand.
4. By 1 and 3 above, a character may only ever wield a maximum of two items at once.
5. When a shield is not actively wielded, the character does not gain the shield's AC bonus (see above - one must "wield" a shield).
6. Page 190, 2014 PHB: "You can... interact with one object of feature of the environment for free,
during your move or action (... you could draw your weapon as part of the same action you use to attack)." This is commonly referred to as making one "Object Interaction" on your turn, though technically note it must be a
part of either the character's (Regular) Action or Move Action (thus Bonus Actions and Reactions do not allow the "free object interaction").
7. For spellcasting: "if a spell requires a somatic component, the caster must have free use of at least one hand to perform these gestures" and also "a spellcaster must have a hand free to access these components but it can be the same hand that he or she uses to perform somatic components" (2014 PHB, p. 203). This means unless a spell has only verbal components, one hand must be "free" (which I think can reasonably be defined as "not wielding any object").
Let's apply this to some specific cases:
1. A two-handed weapon wielder wants to cast a spell on their turn that requires somatic/material components. They probably cast the spell using their (Regular) Action since if they use a Bonus Action for spellcasting, they don't get the "free" object interaction (note this means a spell cast using a Bonus action burns the character's Bonus Action AND their Action by the rules as written). They would probably use this interaction to change from "wielding my sword" to "freeing my hand for somatic gestures, possibly also grabbing a material component" while doing so.
Since the character's hand is now "free" they could not use a Reaction to attack with their two-handed weapon as they are no longer wielding it with two hands and Reaction does not grant a free object interaction until their next turn... when they could use their free object interaction with their Action or Move Action to return their hand(s) to their sword.
2. A sword-and-board wielder wants to cast a spell on their turn that requires somatic/material components. They probably cast the spell using their (Regular) Action. They can either stop wielding their weapon (stow or drop it; to my mind sheathing a weapon is a valid object interaction so I really don't care if you strap it to your wrist to keep it from dropping... you can stow it) or stop wielding their shield in order to free up their hand.
If they chose to stop wielding their weapon, like the two-weapon fighter, they cannot use a Reaction to attack until their next turn when they may interact with their weapon for free with their Action or Move Action to re-draw it (re-wield it).
If they chose to stop wielding their shield, they lose the AC bonus from the shield until they re-ready it ("don it" if you prefer), probably on the following turn - depending on whether your GM uses the 2024 rules where shields don't explicitly have a don time, this could be an object interaction during their Action or Move Action the next turn or if they use the 2014 rules, donning the shield would require your action. I would tend to lean toward the 2024 interpretation myself.
Note in both cases above, you could not use the "free object interaction" with your Action to free up a hand and then use another "free object interaction" on the same turn with your Move Action to re-ready the object, since you only get ONE free object interaction per turn and the second interaction takes your Action (which you probably had to use to cast a spell); I can see a case for taking a Move Action and using your free object interaction during your move to free a hand for spellcasting, using a Bonus Action to cast a spell, then returning your hand to your weapon/shield with your Action, but this ONLY works for spells that can be cast with a Bonus Action (similarly, getting extra Actions by virtue of something like a Haste spell might let you use the second Action to return a hand to your weapon).
Generally, I think the "problem" here ("I want to cast spells and fight at the same time") is that players don't like resource constraints on what they can do and want to be able to use all their cool abilities at once with no downside... and most GMs don't like imposing constraints or imposing downsides ("feels bad"). But by the rules as written a little common sense around the basic meaning of the word "wield" this seems trivially easy to implement and I've been using something similar for years, though not as explicitly codified as the above... and to me, having options with drawbacks where the player gets to select the drawback they want makes for more interesting gameplay, not less.
