A marathon-runner can't... but Indiana Jones could. A marathon-runner can't... but Conan could.
For that matter, why can a fighter, a physical character who is thus somewhat related to realism, go all day, every day - but a wizard, whose powers are completely divorced from reality, can't?
Your plausible-meter could use some adjusting, methinks, if it buys 'shooting fireballs from your hands' and 'being able to fight with a sword all day long without getting tired,' but not 'being able to shoot fireballs every few minutes.' Once you've accepted the first concept, the third is no more or less 'realistic,' whereas the second remains silly in either case - and unlike EITHER the first or the third, it can actually be tested and relates to the objection you voice in the quoted passage.