The orcs will swarm the caster just as well. There will generally only be 1 turn where they are in fireball range but cannot attack. The fireball will wipe out a group, but the orcs can move 90 feet when not attacking and 60 feet when attacking.
There are plenty of things the caster can do to survive. An evoker can blast itself with another fireball. It could put up a wall. It could run and shoot fireballs to delay things a bit. There is always shield, and dimension door can get it to safety. But well over 20 orcs could probably make it to to the wizard, and then bad things happen.
The fighter is more reliable in terms of just dealing damage. It will easily take 1 to 2 orcs down per round, and the orcs will have trouble hitting it and moving through its hp.
My table has only made it to level 11, but I have found that everyone still contributes well to the skills, the combat, and the role play. The thing is, there will always be situations where one class may do better than another, but the group has to work together to survive. It is not a case of each PC fights on its own.