It makes more sense to me to keep the setting consistent and make the adventures fit the setting. No two times that an adventure is run are going to be the same, even with the same Gamemaster running them, so I don't see the harm in adjusting the adventures to fit with a consistent setting. At least that way, the players can be more invested in their characters if they see how the characters fit in with the setting and if the setting can grow and develop along with the characters. This is just my opinion, but I also see how it could be just as compelling to have the characters jump between worlds.