Every game you are playing (not every game system) has a setting in which the activities of the PCs take place. that's what is inseparable from the game to my mind.
Well, I hope you get what you want, in your own games. I'm pretty happy with what I'm doing now, but if I see something someone's doing that I want to incorporate into my play, I'll let you know. It's happened before, so you never know.
But it's already printed the new way. If you're going to go the trouble of re-formatting the whole book, why wouldn't you just do it the way with which your comfortable?
Yeah, this was an area where the, "D&D is now mostly just for new players who aren't used to any other way to organize this stuff" design philosophy hits pretty hard. That was enough by itself to stick with the literally all my other monster books that don't use pure alphabetical.
I have been in many a party that ventured into PvP territory. On a couple occasions I was even an Iinstigator of it, but in none of them was it a real problem in real life.
For me, the second reason is only valid as a reason to skip a check in that instance if the first one is. Fiction or setting logic takes precedence over what would make for a better story (which is my interpretation of, "it would be much more interesting").