I have no problem with DMs making their own rulings that are counter to what's in the book. If you want there to be no non-magical fear in your game that's your decision. I just don't think that that's the RAW, or the RAI, or that it makes role-play bad, or that it's somehow contrary to the way...