I don't think it is really a casters vs. non-casters problem. It is really a casters vs. setting/reality problem.
For instance, a band of fighters attack a dragon hit points are lost or flanking maneuvers employed until one side is eliminated, flees, or surrenders.
A band of casters attack a...