I don't see how this is a problem. We aren't playing a tactical video game here, where every encounter must be perfectly balanced in order to provide an adequate challenge curve for the player over the course of the game. This is an RPG, where monsters exist not where they would be the most balanced challenge wise, but where they would be most likely to exist.
Apart from the usual caveat of "it depends on the game", this is a false dichotomy. The monsters encountered by the party can be selected both to present a good challenge
and based on where they should exist logically (within the context of the game world). That said, the DM can always come up with some explanation for, say, why a manticore could be encountered somewhere other than its favored terrain (if the game even defines this in the first place).
Yes, there should be some sort of balance so there isn't a TPK every encounter, but the reason a lot of people didn't like 4th edition was because it was too clean and perfect. There was none of the messiness of real life, and you always knew what you were getting when you reached a certain level because that happened to be the balanced monster at that level.
I realize that it is apparently not a popular opinion, but I did enjoy the greater precision and predictability of 4e. Not total predictability, of course - even I would find that dull, but a smaller range of potential outcomes. To me, it changed the gameplay by increasing the importance of tactical and strategic decision-making and reducing the importance of luck. It's probably fair to say that my ideal game would be one in which your decisions determine
if you win, and luck determines
how you win.
This mindset is what made magic items in 4E, if not useless, than completely meaningless. "Congrats, you found a +1 Sword! Well, unfortunately at this level, all of the monsters have had their AC raised by 1 to compensate for your magic item, so we can keep it balanced. Yes, this means even the 1 hit point minions will have 22 AC."
As above, this effectively makes magic items meaningless. Magic items should provide you an advantage, and make you feel more powerful, not make you feel like you're just keeping up with the monsters.
Frankly, I think that whether or not magic items appear to be "meaningless" is a matter of how they are presented. The difference is psychological, not mathematical. If I were to change the 4e CR guidelines based on 5e's philosophy, I would simply define CR as the level the PCs need to be to defeat the monsters
without magic items. So, the previously-mentioned CR 12 monsters would now be CR 15 and a suitable challenge for a party of 15th-level PCs without magic items. Then, if the PCs happen to have +3 magic items, they are actually able to take on CR 18 monsters! (CR 15 under the old system.) This would make the magic items seem like they are making the PCs more powerful and providing them with an advantage, right?
The wonderful thing about 5E is that it's flexible. You don't have to throw the exact CR monster at a party to challenge them, you just have to make sure not to throw one so much more powerful it becomes a TPK.
Because 5E doesn't care to provide guidelines on encounters with magic items. In my mind, the encounter guidelines are there to do just as they describe, add lines to guide the DM into making an encounter. They are not there to provide perfectly balanced challenge. They are there to make sure you don't destroy everyone with an ultra powerful monster. Easy vs Moderate vs Hard is meaningless in my opinion, and is impossible to dictate anyway. Consider a brand new group to a tactically minded group that's been adventuring for 20 years. How do you compensate for their knowledge base changing the difficulty of the encounter? I've seen both of these, and a group that knows the ins and outs of the system swings the difficulty into their favor much more heavily than any magic item can. Heck, a string of 20's on a d20 is far more powerful than any magic item in the game, and a string of 1s is worse than the most powerful dragon. But those situations are impossible for a rules system to anticipate. All you can hope is that new DMs won't kill everyone, which is why they are considered guidelines, and why to me, the Deadly guideline is the only one that truly matters.
If that is your philosophy, then I am sure that the 5e encounter guidelines are exactly what you deserve.