Scratch the surface of any "realism" argument and you'll usually find something else at work, something more about gameplay.
Fighters that run out of martial juice just isn't a fun way to play a "fighter" for a lot of people. It doesn't match the fantasy of the class, it doesn't match the story told about why fighters are amazing, it homogenizes gameplay and makes class differences less relevant, it weights character builds over treasure acquisition, blah blah blah.
You're right to be skeptical of arguments about "realism" - usually saying something isn't "realistic" is equivalent to saying it's "bad" or "underpowered" or whatever. It's not really getting at the true friction point. But it gives some directionality. People who complain about "realism" often are concerned with the story they're trying to emulate with the class, and about how the mechanics of limited-use powers butts up against that story. If D&D is at its core a storytelling game (and 5e really carries that banner), the needs of the narrative must be served by the game. If it breaks the story, it won't be a good fit. By "it's not realistic," they often mean "it's not a good fit for the idea I have in my head about it, which means it's not a fun element for me as a player in this game."