There's a bit of a perfect storm of problems.
I entirely believe the stated reasons Rich gives. "Bottleneck" giving them an imperative to fit as much as possible in the books, combined with a mistaken idea that putting story information in there would somehow "prohibit" DMs from invalidating it.
I also think there are additional reasons indicated by the choices made that Rich does not call out, but that are pretty clearly seen.
The bottleneck is understandable, though high-quality IMO generally beats out high-output.
The idea of "negative fluff" is goofy. The first thing most DMs learn is that it is their game and they get to say what goes, so having a "terrain" line in the monster entry isn't going to limit the DM at all. Just because the Blood War is out there doesn't mean a DM can't make an exception in their games. DMs are the masters of their own stories, so it makes little sense to avoid giving them story information that they can easily ignore. If you're designing a game to be played by many DMs in a variety of ways, you ASSUME that most DMs are going to change around information to suit themselves anyway, but you PROVIDE for those DMs who do not. It is not like giving a dragon fire breath is going to give "negative crunch" that a DM is going to feel pressured to include. I can't imagine any DMs feeling very constrained in a game where Rule 0 is the most important and defining rule of the...entire medium of TTRPGs.
There's additional reasons hidden there that Rich doesn't go into, but seem pretty obvious, too.
- "Monsters Exist To Be Fought In Combat": This philosophy prioritizes minis skirmish stat blocks as the most important thing about a monster. It's a philosophy clearly written across 4e. It is a philosophy that is new to the edition, as previous editions included things in "monster manuals" that were not there to be fought in combat, but that were there to be interesting encounters: angels you could ally with, monsters that were more trap than fight, hazards that you could meet along the way, pernicious fey that were mercurial and possessed of dangerous powers...they were books of encounter ideas, not just things to be fought in combat.
- "Combat is the Core Encounter Type in the Game": This philosophy is dismissive of social, exploration, and discovery encounters, and so monsters that exist primarily as these types of challenges are demoted. Even monsters that should highlight theses challenges (succubi should be intense social and discovery encounters) are required to be primarily, and often exclusively, combat challenges. The other encounter types are left up to DM adjudication and skill challenges (which are entirely open to DM adjudication).
- "Screw the Story Material of The Other Editions": This philosophy can be seen clearly in the "elementals - no Archons - no maybe elementals?" elements, or in the "Dryads are now shrubbery monsters" elements. In an attempt to make every creature a combat challenge, and in an attempt to keep "cool words" without having to keep the attached concepts (which don't work in a combat-heavy game according to the theories above), concepts from earlier editions were deemed incompatible with the current edition in one way or another. This leaves WotC free to try to redefine the game as they see fit. Given the above two philosophies, the way they saw fit was largely as a combat generator.
Why don't they have more story info? Like James Wyatt said:
James Wyatt said:
...the detractors have some good points, in my estimation---cute pixies and leprechauns aren't fun opponents, and good-aligned creatures are hard to use in combat-heavy adventures. Yes, people recognize pixies from fairy tales. But D&D is emphatically not the game of fairy-tale fantasy. D&D is a game about slaying horrible monsters, not a game about traipsing off through fairy rings and interacting with the little people.
The philosophies embodied in that approach to game design don't care about much aside from fighting. The Monster Manual showcases this. The class design (attacks and combat utilities are 100% of your powers) showcases this. The fact that Rituals are pointless and that Skill Challenges boil down to "whatever the DM wants to allow" showcases this.
I humbly submit that James Wyatt got it wrong. I think, after 3 years, the current helmsfolk of D&D might be inclined to agree with me: minis combat isn't all the game has to offer.
I hope that Rich notes that while a bottleneck and a shortsighted desire to ignore "negative fluff" might have been part of it, it's not the entirety of it.