What feats do your casters take?
I left off Warcaster, I did take that once too. I took Warcaster after I already had Fey Touched and dissonant whispers and I was playing in a game where the DM rules movement from DW causes a AOO (I've seen DMs go both ways on that).
I multiclass a lot so it is difficult to say "caster" but the last 7 characters who had 3rd level slots took the following:
Human Tempest Cleric7/Fathomless Warlock1 - Fey Touched (wisdom, dissonant whispers), Warcaster
Vampir Shadow Sorcerer 1/Undead Warlock 6 - Shadow Touched (charisma, cause Fear)
Drow Bladesinger 8/Undead Warlock 2 - Fey Touched (intelligence, Hex), Shadow Touched (intelligence, False Life)
Dragonborn Fey Wanderer 7/Whispers Bard 4 - Dragonfear (charisma, forgot about this one too), Wisdom ASI
Goblin Rogue1/Fey Wanderer 13 - Shadow Touched (wisdom, cause fear), Telepathic (wisdom), Gift of the Gem Dragon (wisdom)
Human Bladesinger 9/Arcane Trickster 4 - Fey Touched (intelligence, Hex), shadow touched (intelligence, cause fear), telepathic (intelligence), dexterity ASI.
Another thing the constitution on these characters was all fairly low - the cleric had a 12 constitution, the Vampir had a 14 on rolled stats (and had warlock as her first class), the rest of them had a 10.
Most of the other feats I mentioned (martial adept, magic initiate, lucky, sharpshooter), those I usually take on classes that are primarily martial, although I take Fey Touched a lot on those classes too.