Settings aren't stories, they are a place to facilitate interesting stories. And the way settings are built to facilitate these stories needs to depend on which medium the setting is being made for. TTRPG settings need to be built to facilitate stories
in game for the players. Just like how the mechanics of a game should be built to make the game fun to play, the lore of a world should be built to provide interesting races, characters, and story situations to craft an interesting story for the players at the table. It's a playground with problems
for the players to resolve, not for fictional characters in some novels that progress a metaplot. If there is a really interesting villain in the setting (Erandis Vol, Lord of Blades, Kalak, Vecna, Xanathar, Lord Soth, Asmodeus) that gets killed off-screen by some NPCs in a novel which changes fundamental aspects of the TTRPG setting in future products, that is a mistake in TTRPG worldbuilding. It is not fun for the players sitting at the table when they learn that all the cool adventures were done off screen and the villains they were looking forward to fighting are now dead.
If 5e ever gets a Dark Sun book, Kalak better be alive so my players can kill him. If Eberron ever gets a new novel that reveals what caused the Mourning, that better not be canon to future Eberron setting books. If Sigil is destroyed in a future Planescape game/novel, it better still exist in the next setting book. If Acererak was permanently killed at Chris Perkin's table during Tomb of Annihilation, that should not prevent him from appearing in future D&D products. Baldur's Gate 4 should resolve the endings of Baldur's Gate 3, not the next Forgotten Realms setting book. What happens in D&D companion novels and video games should have just as much effect on future official D&D books as my table's campaigns do.
That's effectively what metaplot is. It's someone else's D&D group or headcanon interfering with how the setting works at other people's tables. Sorry, a half-dwarf in a fictional novel killed Dark Sun's big bad 3 decades ago, we're not allowed to publish his stats in 5e Dark Sun according to
@Micah Sweet. Sorry, it is now 1098 YK in Eberron (100 years later), and everyone knows what caused the Mourning and all the interesting characters are dead. Sorry, Halaster Blackcloak was killed in a Forgotten Realms novel, Undermountain can't appear in future D&D adventures. Sorry, Frodo permanently defeated Sauron in the Lord of the Rings, now our Middle Earth campaign setting book has to take place after all the big problems were solved.
When you read a book or watch a show/movie/play, you feel enjoyment when problems are solved and villains are defeated because you saw it happen. In a game, you feel enjoyment by solving the problems and defeating the villains. If Darth Vader was killed off screen between Empire Strikes Back and Return of the Jedi, that would have been bad writing for telling a good story in a movie series, a sin of filmmaking (show, don't tell). When Emperor Palpatine came back off screen in between The Last Jedi and Rise of Skywalker in a freaking Fortnite announcement, that was a filmmaking sin and fundamentally awful storytelling. If a TTRPG metaplot in any way gets rid of a villain, answers an intentional mystery, or resolves other problems the players were intended to engage with, that is a TTRPG sin and bad worldbuilding. The players should do the cool stuff. The world should be designed around having cool stuff the players can do. Cool villains to kill, cool mysteries to solve, cool locations to explore, and interesting NPCs to interact with. There is no point in doing all of that worldbuilding just to toss it in the trash every time the timeline moves forward.