Sure, it's optional. If you want to reflect the character's race. By default, you are not. Without racial traits applied, the NPCs are just completely generic statblocks. They can represent anyone because they represent no one.
And how is that not exactly what I've been saying?
These statblocks represent NPCs. NPCs do not need Racial traits applied. Therefore NPCs and PCs having different sets of traits is supported by the game.
A change in how they viewed those particular character races? Sure, that's obvious. A change in how they viewed ASIs for PCs vs. NPCs, in general? Not necessarily. It could as likely be them reexamining those races for balance and flavor purposes, now that they were becoming PC options. You'd have to ask the designers to be sure. (Though I suspect they would be rather coy with the answer at this point, knowing the scrutiny they're under.)
NPCs are treated differently in general, but in the core rules, PCs and NPCs have the same, or at least extremely similar, ASIs.
However, if they still thought that NPCs and PCs should have identical ASIs, why have they never released an Errata for the DMG chart?
If they simply redesigned their idea of the monsters, then why not release an Errata to reflect that, if they intended for PCs and NPCs to be treated the same? Why not add the new monster abilities to the new versions of the monsters in Volos?
The Hobgoblin Iron Shadow doesn't have the Save Face racial trait, yet they were released in the same book. Also, for a race known for Con and Int, sure does have a lot strength, dex and Wisdom.
The Nilbog doesn't have the Fury of the Small
The Kobold Inventor doesn't have Grovel, Cower and Beg.
You mentioned that the original orcs had that +2 str, +1 con, -2 Int, right? Here are some fun statblocks to explain then
The Orc Blade of Ilneval has a 10 Int, it is still their lowest stat, with dex at an 11, but I find it interesting.
The Claws of Luthic have 14 str... but also 15 dex, 15 wisdom, and 10 Int.
Orc Hand of Yutrus has a 12 str... but also an 11 Int, 14 Wis, and their lowest stat is actually Cha
The Red fangs have the lowest of the specialty orcs in terms of intelligence with a 9 (the nurtured ones are lower, but they are really more of a side show for the Hands) but an 11 strength but 16 dex. In fact, with a +5 stealth and a 16 dex, they are sneakier and more dexterous than the Drow, Drow Mage and Drow Priesstess of Lolth in the MM.
These orcs definitely all have good Con scores, but two of them have higher dex than their strength, most of them are of average intelligence.
In fact, if you go to the DMG as well, the common orc has a low int (7 lower than any of the volo's orcs except the nurtured one), but the War Chief and Orog have 11 and 12 respectively. Eye of Gruumsh has a 9.
So, 9 printed versions of Orcs, 5 of them have an average or above average Intelligence, two have a 9, which is only a -1, and two have a 7. Does this really look like a race that shoudl absolutely have a -2 Int penalty? The vast majority of their statblocks have at worst a -1 and actually in general are above the average of 10.
I just can't find any support for this idea that we were supposed to have all of these ASIs applying to all of these NPCs. When they don't even apply to all of the monsters.
Of course, because you can customize statblocks however you want. But they only offered one approach to having a NPC reflect racial traits in the Monster Manual and DMG, and that was applying racial traits - either as they appeared in the PHB (with ASIs) or as they appeared in the DMG (also with ASIs).
So... they only designed one way to apply a set of traits for customizing, therefore we are supposed to assume that all NPCs have these traits because that is the only way presented to homebrew them?
Dead wrong actually. You could use the rules from 'Modifying a Monster" on 273. With the Chart of traits on 280 you could then apply many monstrous abilities to whatever statblocks you wanted.
I guess if you want to argue that they only offered one way to add these specific traits that existed in only one form in one place... well, duh? How many other elven abilities would they have tried to give people when elves only had one write-up and it was in the PHB?
Do note though, that when the Eladrin option came out in the Mordenkainen's, the Eladrin monster statblocks are impossible to make with just the Eladrin PC options.
Actually, dragonborn do in fact have subraces, if you count each dragon color. They just all share the same base ASIs. And half-elf subraces were included in Sword Coast Adventurer's Guide (though most folks seem to forget about that book).
In any case, I meant that it would be disingenuous for Wizards to claim that they didn't provide default ASIs for a race on the premise that they had subraces. They were perfectly capable of doing so before. Also, that excuse wouldn't give them cover to avoid providing a default suggested ASI for the base race.
Counting each Dragonborn color as a subrace, when they are not labeled as such, is disingenuous. It also proves my point. Because if you think Dragonborn have multiple subraces under a single statblock, then making a race with a single statblock to represent multiple subraces has been done and is perfectly fine with you.
Or, seeing how similar most of them still are indicates it was just design evolution, while their fundamental thinking remained the same. Maybe it was a shift in their whole philosophy, maybe not. But the first time they explicitly stated that ASIs for PCs did not match the race in general was last year, ahead of Tasha's.
Short of a statement about their intentions between 5E's launch and what they said last year, we can ultimately only speculate. You are perfectly allowed to assume that Volo's was the turning point on ASIs; but I'll need more evidence than that, when there are other explanations.
I think there is ample evidence. Your side is the one lacking evidence. There are few "commoner" "noble" or "guard" statblocks for these races printed. Most of them that are point to my argument. The tables are innacurate, there are multiple routes made available, not just one. Monster Statblocks pretty much never match their PC counterparts in any book.
You can keep believing what you want, but you have no solid evidence that your way was the intended way. We both know it is no changed, and you can't deny a change happened 5 years ago. Whether or not you think that is all of the changes, or they made another exact same change in the exact same way a few years later. Then again here recently... well, I guess that is on you.