It's hard to come up with rules for out of combat activities that work well. They tried in 4E with skill challenges and for a lot of people they were too "mechanical" and seemed to limit the improvisational side of non combat encounters. It was the start of a good idea that seems to have value (much like inspiration in 5E) that just ended up being rather "meh".
Just take a look at the chase rules as an example of what happens when you try to set up a structure,….graft in some other system's rules and I'm not sure how well you could get them to mesh with D&D.
"Grafted" is definitely how I felt about inspiration. And, skills have often felt that way, too, so I can see how Skill Challenges or Chase Rules or the like might not feel like they integrate with the rest of D&D - the rest of D&D being fairly abstract combat, and lots of somewhat less abstract spells, that have applications both in & out of combat.
Skill Challenges were as close as D&D has come to creating a whole-party-engagement structure for the other two pillars, remotely on the same order as cyclical initiative provides in the Combat pillar. While it went from mathematically borked at introduction to functional in the brief 2 years it had, it did not come nearly far enough. And, now that 5e has largely abandoned any but the most cursory attempt to provide structure out of combat, the game's back to a sort of spotlight model. Your character matters - often, as Paul has pointed out, only very briefly - when it has the best (or only, or, at least, a significant/unique) ability applicable to the current out of combat task (not challenge, but specific task, which, often, is a whole challenge). Such resolution is usually quick - the DM narrates success failure, or calls for an arbitrary-DC check first, or narrates the effects of a spell or ritual, and that's it - and involves only the one character. Afterall, it is essentially just task resolution, the equivalent of one attack roll resolving a combat scene.
Since there's no apparent impetus to change that, players are left with choosing classes for their out-of-combat abilities, optimizing skills or choosing spells as the primary ways to achieve relevance out of combat. Optimizing skills, thanks to BA, is largely fruitless. So there's certain classes with specific features that can be helpful in specific pillars, and there's spell/ritual choice. The non-casting Fighter (and Barbarian) sub-classes have nothing much of either.
Sure, the game, overall, could be a lot better at making the other two pillars more engaging, and, yes, that's a much different, much larger issue. The fighter must be judged in the system it's presented in, though, and that's this one.
