Here is how this is handled in some other games I've played a lot.
In Savage Worlds, a successful Test of Wills (Intimidate or Taunt) during combat means that you get a +2 bonus on your next attack or check against the target. (And +2 is pretty substantial in that system, roughly the equivalent of advantage in 5e.) On a particularly good success (a raise; roughly the equivalent of beating your opponent by 10 or more in d20) then your target is also Shaken, which in 5e terms is "incapacitated; save ends."
I like the idea of advantage on your next check; it serves as a good tactical option sometimes, is roughly as powerful as existing 5e options like Hide or Aid, and most importantly, it provides the sort of motivation you want: the guy who just Intimidated you actually is scarier now since his next move is going to have advantage.
In Apocalypse World, there are several moves. Go aggro can be viewed as a combination of Intimidation and Attack. In 5e terms, it's an attack, but you declare something you want your target to do, and if they choose to do that thing, the attack misses. They can choose after you roll so if the attack is going to miss anyway they can just not do it, or if the attack is going to be a crit, that's extra incentive for them to do the thing instead of taking the damage.
The seduce or manipulate move is typically used for Persuasion but it can also be used for Intimidation if you're bluffing. It works very differently against NPCs and players. Essentially, on a success the NPC does what you want in exchange for a "promise" (which need not be an explicit promise -- it can also be a threat, or just an unspoken implication that you will do something). It's up to the PC whether they keep the promise.
When that move is used successfully on a PC, the PC can get XP for doing what they are asked, and if they don't do what is asked, they face some additional consequence. (In Apocalypse World the exact nature of this consequence requires a lot of GM's Discretion.)
The do something under fire move is a sort of catch-all that can also be used for things like talking your way past a guard. "Under fire" means any sort of hazard, and so this move requires a ton of GM's Discretion, and works best when the nature of the hazard is clear to all involved.
TL;DR in other games this is handled by a combination of carrots and sticks. The PCs are always in full control of their characters, but social interaction can give them incentive to comply.