Yeah, that is a primal and fundamental disagreement we're never going to get past, because I see little value in arbitrarily restricting options based on that designers interpretation of the theme, tone or mood.
Moreover, I (perhaps unfairly) hold WotC to a different standard than you, or me, or even other companies like Necromancer or Green Ronin, because they get to use the DUNGEONS & DRAGONS (TM) name on their product. The fact they can put the D&D name on their cover and you can't holds them to a higher level of compatibility with the main game. They need to make a setting work for new players, players who might never own any other book than the PHB but want to play in AL, casual players who just watched Critical Role, and a slew of other non-hardcore players that don't match the tastes of you or I. The easiest way to do so it to only remove things when absolutely necessary.
Only dedicated hobbiests are going to hear about The Lost Citadel (I hadn't until you just mentioned it) so have hardcore dedicated players who love to tinker and decide if X fits in it or not is fine, but WotC isn't going to market Dark Sun to the same audience, they are going to market it to Tommy Newplayer and Gina Streamer, and those two aren't going to be happy finding out half their PHB is incompatible with their shiny new $50 campaign setting book. If you want a Dark Sun that matches the customized ruleset that Lost Citadel, then you have to expect it well sell to the same dedicated hobbiests and the same lower sales numbers. And considering WotC just topped several nonfiction best-seller charts with MToF, I'm guessing they are willing to cede the small pool of dedicated craft hobbiests for the larger market of casuals...
So the question is really, will WotC be willing to accept smaller sale numbers in order to cater to more niche markets, or will they opt to cast the widest net possible even if it means diluting the settings to fit elven paladins in it? I'm guessing the latter. Whether that's good or bad we'll agree to disagree.