Remathilis
Legend
I think it also was done to foster a certain shared "community" aspect to D&D. Most players can tell stories about facing Strahd or saving Waterdeep and this is D&D's way of saying "all of those stories are valid." It also explains why Metaplot is a dying element of world building (because it fosters the notion of a true cannon continuity) and the more "there is no one cannon answer" to a lot of dangling threads (such as what became of Azalin in Ravenloft or the Day of Mourning in Eberron).It doesn’t, but from the perspective of the game’s creators, the idea of there being a diegetic explanation for it is appealing. I think much more appealing than the idea is from the perspective of people actually playing the game. It doesn’t make any difference to me if your campaign and my campaign happen in parallel worlds to each other or not. But I can see how the lead designer (whoever that is now, I forget) might think that having a piece of lore that allows every DM’s campaign to be “canon” simultaneously would seem like something that would be good for the game.
What I truly appreciate is that, unlike some people on this thread, you cannot argue from a strict perspective of what is correct lore and what is not anymore. If I decide to make Drizzt a serial killer, you can't really say my version is invalid. Just different. It's a small thing, but the fact that they have pushed the notion that we're all part of the same Multiverse despite playing different games and using different versions of the settings (or our own personal ones) is kinda comforting.