Because having games peter out over time kinda stinks? Because a satisfying story has a beginning, a middle, and an end?
I mean, yes having a campaign peter out over time does stink!
But I would point out two things-
1. The true genius of RPGs to begin with was the they created a world that would continue from session to session. With the ability to grow and advance. These concepts- the "campaign" and "leveling" arguably helped power the rise of RPGs generally, and D&D specifically, and were truly novel for the time for most people- to the extent that it had to be explained how it was different than the board games people were familiar with (where you don't have a set endpoint, and you don't "win").
2. That said, there are a lot of games out there that don't adapt this model. I like to write my own bespoke one-shots, and there are innumerable amazing games out there that are designed to be run in one (or, at most, a few) sessions.
Again, the problem that I have with the OP is the idea that there is a single "best" way to design a game. It is better to view this as a continuum; the things that work best for some games do not work for others; if I want to play Ten Candles, or Blades in the Dark, or 5e, or Dread, or Amber Diceless, or The Witch is Dead, or PF2, or some other game, I will likely have desires for certain things ... and, conversely, I don't necessarily want the mechanics and features that work really well in one game to be transplanted into another. Although I am currently working on my Ten Candles/Dread/5e mashup, where all the 5e resolutions systems are determined by pulling from a Jenga tower that is on fire.
I'll get back to you on that.