I used to like that idea, but having observed it in practice I've concluded that I want the opposite. Combat, social interaction, and exploration are fundamentally different, and my experience is that a conflict resolution system which attempts to handle all of them ends up doing none of them very well.
IMO, elegance in game design is vastly overrated. Elegance usually depends on emergent effects, which only emerge if the whole system is used as written; this often leads to opaque mechanics whose full purpose is not immediately apparent to the user. In an elegant system, changing or misinterpreting one rule is apt to break a lot of stuff. I would prefer a robust system where you can tinker with one part and the rest keeps on ticking.
I think a lot of what you speak of (and you're certainly not alone in this opinion) stems from maybe 3 issues:
1) The ruleset advice not making the concepts plainly and abundantly clear:
a) Players should understand precisely who their adversary is and both GM and players should understand the stakes and what question is being answered. Its exceedingly simple to discern who the GM is playing if the conflict escalates to combat; the bad guys and the battlefield elements. However, what about noncombat? This is where things go wrong because players and GMs seem to not conceptually get their head around "who the adversary is" ans/or what is at stake (?).
If the player is working out an (i) internal conflict of "I want to learn to control my bad temper" then the (ii) GM is playing "your bad temper" and generating conflict complications that put pressure on your bad temper such that the question (iii) "did I learn to control my temper (?)" is answered.
If a group of players are trying to navigate "Dead Man's Swamp" to find the arboreal home of an ancient medicine man, then the GM is playing the treacherous swamp and the medicine man's hidden home (and perhaps his will to not be found) and generating as complications things that leverage those vectors.
b) Coherency of GM principles and techniques. If the GM's job is to continuously drive play toward conflict until all (overarching conflict) questions are answered then that is what he should be doing. If the GM's job is to engage the PC's thematic content (Beliefs, Aspects, PC-build choices and backstory) and focus totally on that, inhabiting all elements of adversity that interpose themselves between the PCs and the successful resolution of that material, then that is what he should be doing. If the GM is supposed to be generating dynamic complications that incorporate PC decisions, relevant stakes, PC opposition, and an evolved narrative, then that is what they should be doing. Games (typically with conflict resolution at their heart) expect this to be the aesthetic. Other elements brought into play will disrupt this pacing and focus. Incoherency of GMing principles, and/or lack of understanding, is often a primary cause for boring, flat or disjointed conflict resolution.
2) Wonky math or poor synthesis of system (action economy with multiple adversaries can be an offender) and PC build choice + the effects of force multiplication (too much or too little). This can really be problematic when the number of participants is ramped up beyond a certain threshold.
3) User error. This can be due to lack of exposure entirely or due to lack of honed proficiency. Open World Task Resolution is a very different beast than Closed Conflict Resolution. D&D GMs and players who only have exposure to the former and thus try to shoehorn those principles and that aesthetic onto conflict resolution are bound to find the final product disjointed and flat. I just introduced a long, long term D&D GM to conflict resolution the other day and broke it out for him. He was absolutely jubilant at the final product and the dynamic, emergent properties of our tutorial, but it was shocking to him just how conceptually removed it was from how he had run his games (micro-task resolution). The GMing principles shared a certain amount of overlap but there is a mindset and a set of techniques divorced from what he is used to.
Beyond those things, some people just flat out don't like the aesthetic and they don't like the play it produces. Some folks don't consider the zoomed out, abstract process to have enough texture or to be "organic" enough. There is a higher fluctuation of perspective for players so deep immersionists typically aren't fans.
For my money though, a stable conflict resolution system with quality GMing (adherence to principles + honed technique) generates extraordinary emergent play that cannot be delivered otherwise. That is where my interests primarily lie at this point in my gaming life.
