When people say that fighters can't do anything out of combat, or that a caster will always do better than a non caster, it relies on the assumption that the caster will always have the right spell ready, all the time. No one actually has to come out and specifically say that; it's inferred by the argument that is being made because otherwise there is no way that argument could work.
Obviously it's absurd to say that the fighter can't do anything at all out of combat. For instance, he might guard a doorway while the thief works on a chest, or hold a torch so the wizard can decipher some runes on the wall. Similarly, the idea that a caster will
always do better than the fighter is a degree of absolute prognostication that can't ever be true. Afterall, a wizard might lose his spellbook or his voice or something, and do far worse than any fighter until he gets it back. Or a caster might be killed by kobold sling-bullets at 1st level and never raised, thus radically under-performing relative to the fighter at higher level.
Rather, the sense of the complaints you hear is that the fighter has no abilities that allow him to contribute out of combat to the extent or with the significance that other classes have the potential to do. Which is kinda verbose and clumsy, and still not so perfectly complete and precise a
statement of the obvious that one couldn't try to punch some holes in it by willfully misreading and rephrasing it to imply some absurd unrelated claim.
I think the root of the problem is that fighter's cant do anything other PCs couldn't already do.
There is a base-line of things that just anyone can do in 5e, and bounded accuracy makes that baseline non-trivial. Making an ability check or an attack roll, for instance, are things literally anyone can do. You don't need even one level in a class or any proficiencies to make an ability check. Even if you're a 1/8th CR Kobold making a STR check, you'll hit a DC of 15 once in a while.
Fighters are better at attacking (with weapons, without any other special advantage or resource expended) than most other characters and monsters of the same level. Not much better at making an attack roll, but with proficiency in more weapons, combat style, and the ability to make multiple attacks, well, it adds up to some pretty competitive, high-availability, single-target DPR. The fighter thus contributes in combat in a way that no other class can consistently exceed, and most can meat or beat only some of the time. Of course, other classes can make contributions in combat beyond DPR, but the fighter's contribution in the combat pillar is pretty secure.
And that's about it.
Out of combat the fighter can, like everyone else including the 1/8th CR kobold, make any given check the DM calls for with, probably, some chance of success due to Bounded Accuracy. Rogues & Bards can make checks that others couldn't hope to, once Expertise widens the gap enough, Rogues also have Cunning Action, Rangers can track automatically, and I'm sure there's a few other tricks like that out there, but it's rare that anyone can routinely/automatically (without expending some limited resource) make a check that another can't hope to.
So, for any class who doesn't have Expertise or some special auto-check ability, contributing meaningfully, above the bounded-accuracy-ability-check baseline, means expending a limited resource of some kind. Most classes have a non-trivial number of those, typically spells (30 of 38 sub-classes are casters), but also things like Ki points or whatever. Such abilities are often useful in any pillar, depending upon the choices made and how they're managed.
Fighters have two such abilities: Second Wind, which restores hps, thus has non-combat utility in, say, recovering from the consequences of a fall or triggering a trap or something, and Action Surge, which allows an extra action or two between Short Rests (very potent in combat, when the fighter's action can be a flurry of attacks, less so out of combat, when it's going to be some baseline ability check, movement or other action anyone could do - and which doesn't exactly stack up well compared to the Rogue's similar in out-of-combat uses, but 1/round, at-will Cunning Action).
And you have to wonder, if most classes can at least be competitive with the Fighter's DPR some fraction of the time, would it really have been so bad if the Fighter could, some fraction of the time, rival what other classes could do out of combat?
It certainly wouldn't have evoked that classic D&D feel, though.