My belief is that WotC dug themselves into this hole. They did it by believing that perfectly balanced combat was a good goal to strive for.
I agree that D&D is now very much modeled after online combat games, and that's good. However, because of the strengths of the medium, the PCs have a potential infinite amount of resources that they can employ to overcome difficult challenges, including running away, finding allies, doing research, discovering magic items, and so on. However, WotC has created a game where a lot of these things are left nebulous or given little mechanical support, and the biggest focus is just on direct combat with groups of opposing enemies with none of the added nuance in the vast majority of these encounters.
If you are an experienced DM, which most of this forum seems to be, this is not much of a problem. You have so much material, both of your own creation and others, that you can easily fill in these gaps and create truly sublime experiences at your tables. However, new players, which it seems is the vast majority of modern day players, do not have this experience, and the first party material they buy offers very little of this itself.
I think if WotC focused a little less on rigid balance and a little more on giving the party other tools (generators for important allies, spells that could nerf strong monsters, an elegant mechanic for escaping a too dangerous foe, and so on) in their DMG, as well as advice on how to run various encounters of various difficulties in a narrative sense, then 5E wouldn't have near the amount of handwringing online that it does now.
Remember that 5E will never be PF2E, which is one of the most balanced games ever created (and a game I think is very very well designed). 5E has an audience who very much likes interactive narratives and discovery mixed in with their combat. Let's start giving groups these kinds of tools, and tell them not to worry so much about CR and more so about how a group of level 5 adventurers can possibly kill the Ancient Green Dragon through finding a number of long dead dragonslayers to learn ancient wisdoms from their souls and creating traps that will keep the green dragon from flying away and breathing and so on and so forth.