4ed had a lot of good going for it. But the part that was against it was its downfall. Casters.
Yep, casters. They all felt like the same class, fighters with funny names for their powers. The wizards and clerics in particular felt bland and tasteless and guess what? It was the same on the monster's side. Monster spell casters felt bland and tasteless to run. Just like the PCs as casters were bland and tasteless. They all lack the versatility of choosing their spells, the preparation for a definite situation which might not be the same as the one before or the next.
The caster strength is not in the damage it can dish out, it is in the variatiety of the situations it can cope with. Like it or hate it, the spell slot system of BECMI, 1ed, 2ed, 3.xed and 5ed did and still do the job almost perfectly. Be it on the PC's side or NPC's, it does the job. It might looks confusing for the inexperienced, but it is simply a matter of presentation in the MM and nothing else. Had they presented casting foes with highlighted spells useful In combat instead of jamming the selection with useless spells it would have been easier.
I do the exact thing I said with novice DM. I make a photocopy of the mage NPC in the MM and simply highlight the useful stuff for an encounter and guess what? It is almost as if I was a god to them. The new stat block simply do the same but it removes the rest. And that is this removal that makes these NPC special in the first place.
Prestidigitation might not do a lot in combat, but in an RP scene it might do quite a lot. And it is these small details that makes the game so much more believable and poignant. Adding details is not the the first instinctual thing to do when you start DMing and it is why the presence of some spell filler for RP is important. Because for the beginning DMs, if it is not written, it simply does not exists. And the new stat blocks just remove those. And like in 4ed, some foes suddenly becomes bland and tasteless automatons fit only for combat...