I want Morale guidelines, not hard and fast rules. My preferred method would be notes in the monster writeup, "Goblins are cowardly and tend to flee when reduced below half their original number"
This puts it right back in the DMs hand, just like aggro mechanics
But that's part of the question, should "aggro" be a tool that the DM uses in order to battle his party, or should it also be a tool that a player can use to battle the NPCs.
I'm not opposed to aggro as DM fiat, but I do think players should have a hand in controlling the mobs, without an explicit "controller" role.
Here's a fairly napkin-mathy example.
Lets say all creatures have a morale rating of somewhere between 0 and 10.
Lets say that all creatures get X morale points from a particular score modifier, perhaps charisma for players, and various others for monsters.
Then lets say that morale goes up +1 for each member of the [npc] party, with members in a "leader" role providing a larger bonus. IE: the orc war party with the Orc Chieftan has more morale than the orc war party that doesn't have the Chieftan in it.
As members of the group are eliminated, HP are lost(lets keep it simple, 1/2 HP means -1 morale), etc... morale drops.
As morale drops, enemies become less coordinated and more easily controlled by the party. "Taunts" become more effective, enemies engage the closest PC party member or the one who last hit them regardless of how important a target another PC party member is.
Intimidate, Diplomacy, and other checks by the PC party aimed at getting the enemy to give up, then run away or surrender would become more effective. Party morale +d20 roll vs enemy morale +d20(ideally 7 to 8+d20 vs 0-3 +d20). Taunts would function the same, free or minor action, PC morale +d20 vs enemy morale +d20.
High-morale, coordinated war-parties would be unmoved by the fighter swearing at them trying to goad them into attacking him. Low-morale, uncoordinated groups would easily fall prey to his verbal jousting.
As I said, it's a
very mathy example, but the actual execution of it doesn't need to be. The purpose of the whole thing is simply to engage the players
with the DM. Both players and DM would be playing
together not just against each other, and it would provide something to do during combat that's more than just attacking. It could give the fighter more utility, ie: Fighters get a bonus to "taunt" attempts, or characters in the "defender" role get said bonus.
Either way, it engages the players with the dm, instead of simply both whack on each other through proxy.
If you want a more guideliney example:
An Orc war party with high HP and no deaths will not run away for anything, but if they lose their leader or many members they often become uncoordinated and fall prey to their race's mental shortcomings and blind rage. Without a leader, war-party members are most likely to attack the closest target to them, or whatever player attacked them last.