Early D&D was the original version (and inspiration of) the Rogue-like, a game where RNG ruled and the further you ventured, the higher the risk, and the better the rewards gained before you were forced to return to a safe haven. Even if you were aware of your limitations and cautious, there was always the chance of encountering a foe you couldn't fight, and might not even be able to flee from- at which point you create a new character and try again.
Many modern adventures aren't built this way; there isn't a vast, procedurally-generated dungeon to keep trying to push deeper into. There's usually a mission, with set goals. If there is a timeline of events, it's usually pretty generous, to allow for long rests and side quests. The writers might not be overly concerned with the "6-8 encounters" between rests format, and instead you might travel for a day to find only an encounter or two at a location.
As a result, the game encounters problems as it seems designed for a different paradigm, and the management/attrition elements can seem artificial and immersion-breaking, or simply arbitrary. What's worse, the game actually provides players with the very tools necessary to break encounter balance!
Presumably, these were inserted to accommodate an ever-changing campaign, when tiers of play actually meant something beyond "bigger and badder monsters". Once you've become movers and shakers, lords of the land, you aren't really exploring dungeons or worrying about rations for a few days- you're expected to be able to fly/teleport quickly to the far reaches of your domain to deal with threats quickly. Or, instead of reacting to threats, being proactive, and deciding to take the fight to foes on your terms, instead of theirs.
5e doesn't come equipped with many tools or even an expectation that the game will change much over the course of levels 1-20, yet abilities that trivialize standard attrition models and even break encounter design in half are still present, because there was a feeling that the game feels incomplete without them when 5e was being developed.
Think of how everything is tied to an xp budget, that a lot of DM's have dispensed with in favor of "milestone" leveling because it's more convenient to have the players level up at set points, and for the players to be all at generally the same level. I think many playgroups have simply outgrown the DMG encounter guidelines, and the CR system as a whole, which is why it seems to work well for some, but not for others.
What less experienced DM's need, I think, is a book to help them come to terms with the fact that their players and characters can be unique or non-standard enough that these guidelines aren't enough- and to offer suggestions beyond ham-fisted attempts to force them to adhere to those guidelines.