For all these reasons I don't think it's really possible to come up with a firm, cut and dried dividing line between "ok" and "problematic" content.
If that's the case, then the only practical ways Wizards can address essentialism are (in descending order of safety):
1) Wizards has to remove behavior entirely from listings for intelligent creatures, because it's impossible to safely avoid essentialism
2) Wizards has to include multiple, distinct, equally valid versions of every intelligent creature's behavior
3) Wizards needs to affirmatively declare when an intelligent creature's behavior is a matter of fantasy-biology or a matter of culture (and readers need to accept that either option is fine; which is the tricky part)
4) Wizards needs to clearly and repeatedly explain that the described behavior for every intelligent creature is provided only as a convenient default for game purposes, and should not be taken as required traits for every single member of the species (and readers need to understand and accept this; again, the tricky part)
The idea that there is a normative default with abnormal exceptions is what makes those characterizations essentialist
What do you call the redcap description in the OP, if not a "normative default" for the species? I don't see wiggle room in there, as written.
So you can either accept that the redcap described is an example, from which you can deviate... or it's essentialist.
I gather that you think this is not a justified reading. I get the feeling that wizards, and maybe others in the community, don't have a strong interior sense of why people respond to the various representations that they/dnd puts forward as problematic or not. And so, it becomes like a guessing game, because the things that are identified as offensive to some seem almost like random selections to others.
If it's impossible to distinguish between essentialist and non-essentialist descriptions of monsters, then you either have to adopt some version of the approaches I suggested above, or you simply ignore the problem until someone specifically calls you on it, which isn't a great strategy for several reasons.