True, some people like to think of game rules as like the laws of Physics, they describe how the world really works. And 3rd edition leaned into that, whilst the other editions all went the other way.
What I find most humorous is that this is now the prevailing opinion, but back when 5e launched, I very much saw the opposite--folks felt D&D had always been at least somewhat simulationist and 5e was finally returning to full-throated simulationism. The despised gamism of 4e had finally been slain.
It's just funny how the tides change.
If the NPC is being designed to be a common element of the world, then it should not have abilities that are not common elements of the world.
Even if I granted this if-then, my dispute is the antecedent, not the consequent. I don't accept that a creature labelled "Apprentice Wizard" is meant to be some Platonic ideal of all individuals undertaking wizardly apprenticeship. I don't accept that a creature labelled "Green Recruit" is meant to be some universalized representation of absolutely every being that can be parsed as a fresh, inexperienced conscript. Since I reject the former,
even if this conditional were true, it would not establish the truth of the consequent. Of course, it also wouldn't establish that the consequent were false, but since this "creatures should not have abilities that are not common elements of the world" claim is your assertion, the burden of proof lies on you. I make no special assertion about it.
Further, even if I granted both the conditional
and the premise that NPCs are or should be designed as common elements of the world, it wouldn't follow that this Platonic idealization actually does result in your desire, the "taking the diegetic context it occurs in seriously." Because enforced uniformity is (or at least seems) just as artificial and contrary to diegetic seriousness as the (allegedly) artificial situation you're complaining about. That is, why should it be the case that
absolutely all "Apprentice Wizards" learn exactly the same stuff in exactly the same ways? Real, playable wizards don't work like that. You can take zero spells that do damage, if you want. You can take
only spells that do damage. You can hyper-specialize in spells of only one or two schools (depending on which school, some are more dense than others.) You can ultra-generalize, intentionally avoiding over-emphasis on any narrow set of schools. Etc.
Likewise, why should the "Green Recruits" of Yuxia, the Jade Home, have any meaningful resemblance to the "Green Recruits" of Al-Rakkah, the Jewel of the Desert? Sociocultural differences, economic differences, fighting style differences...there not only could but
should be a mountain of differences for why one group might favor archery and horse-riding (as the steppe barbarians do) over heavy armor and metal shields (as the remnant population to the south does) or lightweight armor and "CURVED. SWARDS!" as the blademasters of Yuxia do.
Enforced conformity and uniformity has just as many diegetic problems. And if we aren't enforcing uniformity...why should we expect every
narratively "person who is green, and also a recruit" to always, 100% of the time, use the first creature statistics published under the name "Green Recruit"?