Fiddly and/or unique abilities are almost the opposite of what I'm talking about, let alone coming up with them on the fly. The system should be able to able to model a reasonable range of combat and noncombat tactics such as "I try to grab the orc's spear out of his hands." Or "I try to make the hobgoblin angry at me by insulting his paternity." The system should offer viable alternatives to "hit things" that are open to everything. If the GM feels these tactics aren't viable, they should houserule ways to make them viable. And then apply them consistently to both monsters AND player characters.
It more or less does…for players. For monsters they kneecapped the ability to do stuff like that on the fly, and anything even remotely resembling a ‘taunt’ effect against players has almost always been rejected, doubly so for nonmagical ones.
4e went the completely wrong direction by squelching any non-hit things action and then designing a horde of fiddly, inconsistent, unique abilities for every monster based on breed or military rank. Many of which accomplished similar things in highly inconsistent ways. Action types that should have been open to anyone were monopolized by specific race or class options for no justification whatsoever beyond "balance". 5e hasn't done a fantastic job in that department by comparison, but there ARE at least base grapple, shove, aid maneuvers as well as more expansive (if less defined) skills.
I suspect ‘fiddly’ and ‘inconsistent’ are editorials, since most of the monster abilities were just riders, AoEs, or Save Ends effects, none of which are what I would call fiddly, especially since 2 of those are present in 5e as well. You’re also going to have to elaborate on what action types were unavailable to everyone, since you can definitely make basic attacks, bull rush, grapple, and aid another without needing class support to do so. Whether or not you were good at it was a result of other factors, but that’s pretty much the same story with 5e, a wizard can grapple, yeah, don’t expect him to win very often. The only thing I can think of off-hand is a marking mechanic, which is only vaguely even in 5e at all, save the more recent UAs.
Also, you seem to be going back and forth between player options and monster options, when we don’t care about the former in this discussion. This is about monster design, not player/class mechanics.
Wrong. First of all, these aren't connected concepts. Skills are hard-coded monster abilities that provide tangible benefits, if ill-defined ones. (And the ogre doesn't get any beyond base stats btw).
Not wrong at all. All of that stuff you mentioned earlier about grabbing spears and insults and whatnot always boils down to a skill check if it’s not an explicit part of the monster’s statblock. So yeah, you end up RP’ing the choice of action, the efficacy of which is determined by the skills in question, or default stat bonuses in this case. Which is also an excellent counterpoint, even if there was a suite of more robust combat options, it’s still possible that the ogre ends up being a boring auto-attacker by virtue of not being effective with the new options.
RP is inappropriate for gauging class or spell balance since that's character dependent, but it is entirely appropriate for monster balance since it determines monster behavior. For example: If you take a base ogre as GM and give it a dozen contingency plans or play it as deftly manipulating the labyrinthine schemes of Drow nobility (beyond brute intimidation) then I'm sorry but yes, you're doing something wrong. 4e absolutely could have fixed their base system by offering reasonable non-hit things options, 4e GMs could have houseruled their way out of the problem. But the base system made it much more difficult to do so.
Not in the slightest. I could probably think of a dozen reasons for an ogre to be intelligent, from finding a headband of intellect, to divine meddling/lineage, demonic possession, being the subject of arcane experiments, the list goes on and on. Outside of combat there are so many dizzying factors from campaign theme to setting to character intervention that those options are best left lightly touched on at best, since there’s no assumption that they’ll be used. There is an inherent need for the Ogre to be usable in combat, though. 4e design was great about this, giving a slew of background info on how a lich operates is of little value when they tend to be used sparingly and customized as overarching villains, but having them perform well in combat was of utmost importance when it came down that. I’ll definitely say they didn’t hit the mark perfectly out the gate, but the reasoning was solid.
As a GM, a document that provides ideas for RP, social perspectives, religion, hierarchy, and other things to inspire the imagination is far more valuable than wasting space on separate stat blocks for ogre bludgeoneers, thugs, skirmishers, warhulks, and other pigeonholed combatants with slightly different circumstantial boni.
All those NPC stat blocks at the end of the MM and VGtM are designed to be adjusted to an assortment of monstrous races. Need an ogre shaman? Fit the ogre stat and special ability adjustments to the Druid or Acolyte NPC block. Want to create a swashbuckling gelatinous cube for some tortured in-game reason? Gelatinous cube stats and abilities mixed with Swashbuckler (VGtM). It's a pain to recalculate CR in 5e, but the tools are there.
Pretty much shot your argument in the foot at the end there, because yeah, you can painstakingly craft every foe to have some modicum of options or abilities, but the crap CR system makes it a pain to do in a balanced fashion and it takes up precious planning time. When I buy a product, I’m expressly doing so for that reason; to have to the pros craft interesting monsters so that I don’t have to. Also, given how bounded accuracy works I don’t think we have to worry about 30 different ogre variations, but we could certainly use a table of alternate actions/abilities to add as needed.
Same chapter different paragraph with the RP, there’s an endless wealth of materials and settings both within D&D and fantasy at large that you can draw from to get background information for ecology, the exception of course being if you want to play in an established setting. That’s my problem with the 5e lore, is that in addition to eating up space that could go to statblocks or art, they waste a bunch of time with lore for a setting I’m not going to use in a product that really should go out of its way to be system agnostic. Doubly so when products like VgtM come out, which is exactly where that rich setting RP stuff should be revealed.