And we reduce that complex interaction (in most cases) to a single roll, or perhaps a contested pair of rolls.
How un-fun would RPG combat be if it were resolved in a single roll? I'm not sure I'd even play RPGs at that point.
But at the same time, I can't really imagine what a complex system of rules for social interaction would look like. As you said, we are using dice rolls to represent sword swings, but that's partly because we aren't actually swinging swords at the same time. Replacing something we are capable of doing...talking...with abstraction makes the game less rich, not more.
And even if we keep the talking, but what matters is the dice rolls, then the talking feels...pointless.
It just occurred to me that I treat social interaction not like combat, but like traps: I have two kinds of traps: those that are hard to find but easy to avoid, and those that are easy to find but hard to avoid. In the former case I telegraph the presence and it's up to the players to read the clues, and in the latter case I leave the solution to their imaginations. In both cases I consider it a success if no dice are ever rolled.
What I find boring (and honestly, in my opinion, "boardgame-ish") is:
"I roll to search for traps...17"
"You find a poison needle trap"
"I roll to disarm...23"
"You disarm it."
I don't want that to happen in social interaction, either.