WARNING: Minor Critical Role season 2 spoilers - I don't know how to do the collapsable spoiler thing so skip this post if you care about Critical Role spoilers
I only DM at home with a group of players that I've been playing with for over two years. It is easier when you know your players. If you go too far or your players do, it is easier to call each other out without too much embarrassment or hurt feelings. Make sure players are comfortable telling you when they are uncomfortable. In public forums, when playing with strangers, you have to be more careful because you don't know everyone's personality and life experiences. You can't anticipate every issue someone might have. I guess were I to DM in a public venue, I would keep it PG and not be defensive if someone expresses discomfort.
When starting a new campaign or bringing in new players, I will mention a few ground rules, but they are not black and white.
1. No evil player characters in my campaign.
I'm just not interested in that kind of game. That said, I do like to put characters in morally ambiguous circumstances. At times they will have to question if they ARE the good guys. But I have no interest in participating in the role-play of torture and rape, for example.
2. Sexual situationsif they occur, are "fade to black" situations. Look, if you want to play the Cassanova bard who uses his or her sexual prowess to woo people to gain information or just for fun. Fine. But I'm not roleplaying a sex scene with you.
3. Violence against children.
I'd probably just scrap the campaign if I had players killing kids for kicks in my game. But, at the same time, there could be some interesting dilemmas to role-play here. What do you do with the kids of "evil" races when you clear out their stronghold? Similarly, having a terrible beast harm or threaten to harm a child is a common trope because it can make the characters become invested in stopping the evil and more willing to take risks to save / avenge the victim.
I just listened to a recent episode of Critical Role where this came up twice. First, Matt Mercer graphically described a child being mauled by a gnoll. The second instance was when the party killed a baby manticore in front of its mother.
As for the scene where a gnoll mauls a kid. It was borderline disturbing for me, but it was supposed to. It wasn't gratuitous and it is not like Mercer fills his sessions with graphic depictions of violence to kids. I think that it was appropriate in this context. It added to the game. It seems that some players, especially those in their teens, like to be shocking for the sake of trying to be out-shock each other. That's not the case here and I think it can be appropriate. But if I had someone in my party who recently lost a kid, or someone who had been mauled or had a child mauled by a dog or other animal, I would be sensitive and try to either avoid such scenes or just not go into graphic detail.
As for the second, it felt a bit like the player WAS trying to be shocking for entertainment value. I rolled my eyes as I listened to it. I would get annoyed as a DM if there were players acting out the whole "hey bad guy, watch me kill your children." I though Matt Mercer's response was interesting. I didn't expect him to chastise or say no, which he didn't (though there was some groaning and "really?" comments). Instead he gave the mother Manticore advantage on all attacks against that player's character and it ONLY attacked that player's character, ignoring all the other characters. While I rarely allow "narrative sense" to dictate my game's mechanics (I'm not playing for an audience), I would definitely consider doing this.
On the other hand, that could be the form of feeding the troll. Doing so just gives that player the extra attention they seem to be seeking.