It's a deeply rooted belief, concept, trope, or whatever that all around competence goes up with combat experience. Han vs Luke at the beginning, Aragorn in general, the way the hobbits got better from their exp in Moria. Combat experience does something special to people that you don't get otherwise. So you say what about parallel leveling systems- one for each of the pillars. Or, maybe a system where you have to get exp in each pillar to level. So that's where it gets too complex, so in order to utilize the other pillars needs good design.
Regarding downtime activities. Certainly training is necessary. Irl is boot camp. But it's the application of that training in combat that is necessary, else no one would do combat. Irl, no one wants to be in a combat because no matter how experienced you are, it is always dangerous. So, irl, everyone will try social or other skills to accomplish the task rather than engage in combats. So in game, we have the essential contradiction that the main reward is level up through combat. In past editions, they used gp as exp to push non-combat solutions. The de-emphasis on magical items is also a problem. Christmas tree characters were a problem, but now the problem is there's no reward for non-combat. Also no lure to dungeon delve. Smart characters would level up on random wilderness encounters to a point where the dungeon with +1 items has super easy encounters for them, then just plow through that for the items for overcoming resistances, then never dungeon dive again.
Anyway. I don't know the fix, but I do know that getting better at combat from not combat goes against some basic, deeply rooted instincts that all the best fantasy and adventure literature is based on.