hawkeyefan
Legend
To have clear mechanics would require a considerable degree of complexity as a huge variety of situations would have to be accounted for. Doable, yes; but the returns may or may not be worth the effort.
It would not require that. The rules can be as simple or as complex as one wanted, but it would not be required to be either. I would think the simpler the better, but that's with 5E in mind. Other editions of D&D or other games might call for more complex or less complex rules, and of course any play group will have their own preference.
In any event, if the returns were not worth the effort, then I would say to lower the complexity to a point where the returns become worth the effort.
It could be something as simple as a kind of reverse morale roll. At signs of the enemy retreating, make a morale check and have tiered results like "total pursuit/attack" and "parting shots" and "celebrate the victory". Something like that.
Indeed, it comes down to the DM playing the opposition in good faith and as far as possible thinking as they would.
Sometimes the foes will chase fleeing PCs if it's to the foes' advantage to do so (almost always the case when, say, the foes can fly and the PCs cannot) and-or the foes know they have the upper hand and are smart enough to realize they need to strike while the iron is hot.
Sometimes the foes are just glad to see the PCs off (and almost always the case when the foes know they cannot outrun the PCs) as - unknown to the PCs - they too were barely hanging on, and-or they've defended what they need to defend (often, their homes) and see no reason to move away from its location.
Sometimes it's a mix.
Yes, but this is where the GM has significant input. You say playing the opposition in good faith....but faith to what? To the goals and ideals that the GM has given them. The GM has also decided the types of enemies and their number and very likely the information that brought the PCs to this encounter and on and on. Perhaps the players got themselves into this mess, that certainly does happen, but very often the GM will have a very significant role in getting things to this point.
Again, if the goal is to make party retreat an option that players are more likely to consider, then barring mechanics for this, the GM's response when PCs do retreat is pretty much the only factor. So in that case, I'd say that in all but the most extreme examples, let the PCs flee. Have the NPCs gloat and mock them as they run off. That's a perfectly valid response.
Now, you don't want to always have that be the response, I get that; having the same outcome every time makes things predictable, and that works against the sense of danger that should be present. But if you take a GM who always has the enemies allow a retreat and a GM who never has the enemies allow a retreat, I think the latter is worse for the game than the former. It's no less predictable, and it essentially removes a party retreat as a viable option.