A D&D edition change is usually much more seismic than a "periodic rules update," or else we wouldn't have had the 3e/4e edition wars, and we wouldn't still have some very vocal 4e fans complaining about what they see as a kind of design backsliding in 5e. We wouldn't have the OSR trying to recapture a simpler, deadlier era of play. We wouldn't have had 3e's "back to the dungeon" philosophy. We'd be able to use the OGL to publish 4e material because 4e is just a "rules update" to the same fundamental game.
This ultimately goes back to the diversity of audiences that D&D serves, and how different design works for different groups. Invested fans are protective of their version of the best D&D because the official vision of what D&D is can be described as incoherent at best. So fans have to plant their flags and get invested in the rules that work best for them, since there's no guarantee that the official releases will actually still care about their style of play in 5-10 years, and some very robust evidence that, in fact, they will not.
(and, as an aside, the tendency to just re-use catchy names on unrelated design does exacerbate this problem)