Let's reference some things that WotC officials have talked about.
Crawford has mentioned in previous interviews that games tend to last 3-4 months, and that they are generally bound between Levels 1-10. There are enough games outside this paradigm happening to make Levels 11-20 worth keeping around, and clearly people have longer and shorter games (I have two campaigns I'm running now that are a year+ anecdotally). However, the average being 3-4 months and about 10 levels makes sense to me.
Running a Sandbox in 3-4 months is certainly doable. I think a lot of people have run games like this, especially people who grew up with older editions. But I'm not sure if it's more satisfying then having complete an adventure in 3-4 months. I've run and played in lots of sandboxes (homebrew or otherwise), and games that have a story that actually finishes feels like a white whale. And that's because for a long time, I didn't focus hard enough on keeping my campaign constrained given the realities of the people I was playing with. Especially when most of my games are online, people just don't carry the same investment + it's a lot easier to have a messed up schedule. As a result, I've turned to doing more "story arcs" for groups so that campaigns can end, everyone gets to experience that sweet dopamine release for completing something, and then I can build on that in the future if the group can continue.
Let's just ASSUME my experience is a common experience. Or that I'm one of many similar-but-different experiences that regardless still contributes to this average of "3-4 months." If that's the case, and I'm WotC, and I want people to feel satisfied from my adventures (because we turn to stories for emotional experiences of various kinds), I'm going to focus primarily on short form adventure style. And by shortform I mean anything from a single adventure in an anthology to a full book going from levels 1-10 (because things are on a spectrum, not black/white).
My favorite adventure book, despite being one of my most loathed adventures, is Descent Into Avernus. I love this book because it equips me with the tools via the Baldur's Gate Setting Guide to create my own 1-10 adventure, and enough "models" in the pretty mid core adventure that I can create my own "Go to Avernus Hell story." More and more, we are seeing this focus on setting material and treating the book as a model being used in addition to the arthrology models that are pretty much the same thing but better executed. Now that the DMG24 encourages style of play, we can safely reason that WotC has decided that chasing that average is better for its game and for the company's financial health.
I like it too because now D&D is taking a more concrete shape. This is an action-adventure story game with a moderate amount of mechanical heft. Classes are developed with story in mind. Stat blocks are developed with story in mind. Adventures are developed with story in mind. The designers repeat this so often in interviews. D&D is a game about telling stories that combine action, humor, sentimental emotion, and a rain check for anything else you want to bring in (post-apocalyptic Dark Sun, horror-based Ravenloft, Great War Dragonlance, Indiana Jones Pulp Eberron).
Now, this doesn't mean a sandbox can't be used to convey a story. Tomb of Annihilation is something I have actually played through beginning to end and yes, it's a true-blue sandbox. You have to fill in MOST of the hexes yourself with the materials given or whatever else you want to add. And the pieces that are given are specifically designed to tell a story if a DM doesn't have the time, energy, skill, or resources necessary to do the full sandbox.
This focus on story combined with adding a special factor dependent on setting or adventure IMO is a winning formula for 5E in terms of quality game support. They are appealing to what their customer base LARGELY wants, and giving peaks into other styles of gameplay by figuring out how to mold story with the accessory genre. It's the same strategy the MCU used to become successful (spy movie Winter Soldier, horror movie Multiverse of Madness, Western Logan, but all still comic book action movies with a bit of humor and sentimental emotion).