I have only limited experience with even-numbered editions of the game; most of what I've played have been BECMI, 3E/3.5E, and 5th Edition. And I was never a member of any game design team, so I can only speculate on their motives and directions. So with all that said:
2nd Edition felt very different from BECMI...there were a lot of changes, and a lot more complexity. And that wasn't surprising; this was the whole point of creating an "advanced" product line. But all in all, it felt like these two had more in common than not. I didn't like the updated game mechanics (I felt they were too complicated), so we never played it much. I really liked how all of the lore was expanded and the campaign settings that were released with it, though, and I still cherry-pick from them today.
3rd Edition felt drastically different from 2nd Edition AD&D. Also not surprising, because it was being produced by a different company with different goals in mind. The third edition felt like a whole new game compared to BECM (which I was still playing at the time) and AD&D (which I had only played a couple of times and didn't prefer.) We switched from BECMI to 3E, and played it for more than a decade. We really liked the modular rules and unified d20 mechanic, and the immense amount of supplemental materials.
3.5E felt nearly identical to 3rd Edition. Not surprising either, because it was intended to be more of a "bug fix" or "patch" and not a whole new game. We switched to 3.5E in mid-campaign, and barely noticed.
When 4th Edition came out, it felt a lot like when AD&D 2nd Edition was released: there were a lot of changes, but it still had more in common with 3.5E than it didn't (especially if you were playing with the Book of Nine Swords expansion). And like the 2E AD&D game, I didn't care much for the changes to the game mechanics. (I know that there is strong disagreement on the amount of influence that MMORPGs had on the 4th Edition game, and I know where I stand on that topic.) When our 3.5E campaign ended, we decided to switch to Pathfinder instead of 4E. But between you and me, I thought 4E had the better lore and flavor.
5th Edition somehow felt both like a step back from 4th Edition, and a step toward BECMI. If that makes any sense? It recaptured a lot of the old-school D&D feel that I had been missing from BECMI, mostly with the tools that the DM was given and how the DM was expected to adjudicate things, and I really liked that. It was also cleaner and more streamlined than 3.5E, and I really liked that too. So we switched to 5E, and I doubt we will be changing again anytime soon. I just wish they had kept more of 4th Edition's lore.