Yes! Very succinct. I think the smarter the monster the less the need to determine its decision-making with random rolling.That's a good observation. Don't let the barbarian go around casting quasi-sleep nor quasi-fear. The barb can Charisma check (or Intimidate, for you skill-lovers) one opponent, the situation can Charisma check everyone. Further, making an opponent run away with Charisma counts as an action, because if it's free, then it's even better than quasi-fear. For results, see below.
Which DMG? Since you're playing D&D 5e (or OneD&D, which "is not a new edition"), you don't want numbers stacking up. That was 3rd ed. If you have to roll, use Advantage or Disadvantage as appropriate.
As far as when to roll, I'm thinking you can ask yourself two fairly simple questions when they seem appropriate. As the NPC:
- Do I think I'm going to die if I keep fighting?
- Is the reason I'm fighting worth dying for?
Which brings us to another question - should heroes be able to make a skill check to dissuade enemies from retreating as a reaction?