I would agree.
What I and others are failing to understand is why you can't just say "this guy is not going to be intimidated by a few words in combat" and let everyone move on with their lives? Because you've got a player who thinks they're playing 4e and you've taken away their power card for no good reason and ripped it to shreds in front of them while laughing maniacally and collecting their tears for your immortality ritual? That hardly seems like anybody else's problem but Tom's. Ugh, Tom. Tom! It's always Tom, isn't it?
5e empowers you to rule in exactly that kind of way, for what it's worth. Your choice to disregard that is your choice, but recognize that it is not the default one.
And if you, as the DM, feel that you would be entirely unable to make that ruling, then there's nothing I or anyone else is going to be able to say to you. You see a duck and only a duck. You're playing a fundamentally different game from the one we're playing, one where the DM has no agency in dictating the rules and internal consistency of the world they're running and is instead enslaved to the players' interpretations of the RAW.
Maybe that's an artifact of AL play, and if it is then thank Arawai I've never subjected myself to the AL. That's not the way I play the game, nor would I want to participate in such a game.