Like Zardnaar said on page 1 the economics encourage edition changes all on their own. There is another reason however that I believe substantial edition changes are necessary to D&D specifically. That reason is what I'd call the funnel. The thing that makes other rpgs that have a more iterative design work is that they cater to a specific fanbase that groks what they are doing. D&D is deferent, it's overwhelmingly the first, and often the last, TTRPG someone plays. There are so many different ways that you might get interested into TTRPGs - from books, to board games, to t.v. shows, to video games as diverse as Baldur's Gate, Elden Ring, Disco Elysium, The Legend of Zelda, and Final Fantasy 14. All those possible entry points funnel down to the current edition of D&D. This results in a in a pool of players forming a very diverse amorphous blob that pulls, pushes, and stretches itself in countless and often contradictory ways. That metaphorical blob does occasionally form consistent patterns over time though, as the tastes of the nebulous 'overall TTRPG' community changes. This is why I think D&D specifically does need to periodically change fundamentally. Sometimes the tastes of that blob just gets too far out of step with the game that exists.
Now you might say that the problem is the funnel itself, and there's some truth to that. The thing is that many players have difficulty learning a new system, or even learning their first. Having a go to game also greatly increases the chances that there is a game locally that you could join. Finally, Hasbro definitely likes the funnel and has every possible motivation to keep it going. So I think the funnel complication is something D&D is just going to live with for many years and even decades to come.
