When exactly did this paradigm shift happen with DMs? Where it went from, "If you're the DM, it's your world and you need to prepare the adventures your players will take their PCs on, and control all the monsters and NPCs as living beings" to "I just want to put pieces on the table and roll dice as the DM, and can't be bothered with learning powers/spells/abilities/rules"
Because that's what I keep hearing from folks like CapnZapp. The refusal to put in the work required (and I'm not talking hours "perfecting the evil overlord's plan") of a DM is NOT a failure on the design team, like you keep saying. As the DM, you have a LOT of power at your hands. Literally everything about the game has to go through you, which includes role-playing monsters/NPCs like they would normally act. If you (general you) don't want to use the role-playing elements of the game as a DM, then maybe you should be playing a boardgame instead, or don't DM. The trade off for having all that power is the responsibility of preparing. It's the DM's job to make monsters interesting, and it's not that hard. All you have to do is play them as they would normally behave, which includes behavior outside of the immediate combat round. Because if you don't, then all you do have is nothing more than a boardgame. What are the enemies doing in their normal day to day life? What would they do if they discovered PCs lurking about? etc, etc. Kobolds are certainly intelligent enough to utilize things like ball bearings, burning oil, baskets of snakes, traps, etc and get those things prepared before a battle. Do you really need those things listed in individual stat blocks when most everyone else has already figured that out? I certainly hope not, because that's limiting by implying that kobolds can ONLY do those things, when the truth is they are only limited by what you can come up with as the DM that is appropriate to their intelligence and/or ability. Which should be noted is a lot more stuff than can fit in a stat block.
And if you want an answer to "how did the designers end up here", the answer is pretty obvious. Because most players didn't want and/or need redundantly listed abilities that resulted in full page stat blocks and increased page count, and didn't want and/or need everything a bad guy could do spelled out for them or they felt restricted. Sales #s of the previous edition and the current edition seem to back this up. And from a purely business perspective, the designers seem to have given the majority of players what they wanted and put together a very highly successful game, so that means they did a helluva good job, regardless of what your or my individual opinions are.