To me, the beginning of the end of the Gygaxian Multiverse was, weirdly enough, the Manual of the Planes, which was devised and released after Gygax was ousted. A lot of people really liked it, but the problem with that book is that it was 128 pages of detail about the various planes .... and do you know what it devoted to the infinite multiverse of Prime Material Planes?
.... three pages. In an APPENDIX. And it reduced the glorious diversity to three attributes that the DM would roll-
The Physical Factor (things like sentience is impossible, because all matter reacts with other matter and explodes, or nonsentient items like chairs are fully aware).
The Magical Factor- (from universal spell casting for all sentient beings limited only to their imagination, to no magic or imagination or creativity can exist).
The Temporal Factor- (weirdly, not all about time! It's really how closely linked the plane is to your prime plane; so it can go from very different because things are all different colors and planets lack atmospheres to the plane is similar to your prime, but millions of years in the past).
Importantly, it was boring, not helpful, and travelling to other prime planes had a decent chance of killing you from a random role of the dice.
So the book was basically, "Look how cool the outer planes are, but the primes? They suck." And that carried on after that.
I would argue that Gygax mainly thought of the outer planes as sources for stuff to come into the primes- devils, demons, solars, and so on. For him (and others at the time) the outer planes weren't interesting, because you had the infinite variety of the prime material multiverse. Both at his home table and in published adventures, he was constantly using the wild variety of the multiverse.