Because you don't actually understand what was wrong with the original products to begin with. Al-Qadim is the closest to being alright, but even it has some serious stumbles. There is nothing wrong--nothing inherently "problematic"--about writing a setting that is inspired by non-European cultures, mythologies, or histories. The guys who made Avatar: the Last Airbender, which is very clearly and intentionally drawing on Eastern cultures and ethnicities (the Fire Nation is fascist Imperial Japan; the Earth Kingdom is like Han Dynasty China; two of the three Water Tribes are almost explicitly Inuit peoples; and the Air Nomads are Tibetan Buddhists, alongside stuff like the Sun Warriors who are clearly Mesoamerican-like), yet both of them are middle-aged white dudes.
The difference is that they did the work to make it respectful and serious. They hired legit actual experts on many different topics, including getting people who could actually write both modern and ancient Chinese so that written documents would, in fact, be written correctly; they had an on-staff martial arts expert, Sifu Kisu, to whom they dedicated an actual in-world character (Master Piandao); and they had folks actually critically analyze their work.
In the 70s and 80s--hell, even in the 90s, albeit not quite as badly--foreign cultures were used exploitatively. It didn't matter what those people actually thought or did, what their histories or beliefs or practices actually were. It just mattered that it had the veneer of exoticism, of foreign-ness. Kara-Tur is, unfortunately, rife with Orientalist tropes and content. They were not "problematic" because they examined a culture other than the authors' own; they were inconsiderate (or, more commonly, outright disrespectful) because they treated those cultures like "exotic"/silly/strange caricatures to be pantomimed for a little while and then set aside.
My frustration is that we have a self-perpetuating cycle. FR is the most popular setting, so little to nothing gets published for anything else, so FR is the most popular setting, so little to nothing gets published for anything else, so...
D&D fans pride themselves, almost to the point of hubris, on the fact that D&D is unlike video games or even most board games, by being allegedly open to the entire panoply of human imagination. Anything you can possibly conceive! But instead we grind endlessly on the same incredibly tired repetitive concepts, with the only meaningful variation in the past 50 years being the degree to which you ascribe to cynical realpolitik or idealistic monarchism.
I want these other things to be shown off in the core books, to be actively engaged with in the DMG, to be published in core materials, because that's literally how we break this self-perpetuating cycle. We get people more interested in more diverse ideas by SHOWING them how to make use of more diverse ideas. And in the doing, we make not just D&D but TTRPGs in general richer, better, fuller--because they embrace more of what it means to be human...or something other than human.
Do it yourself then. I didn't wait around for TSR or WOTC to do it for me. I did my own research and when I use these places, I alter what stereotypes I know of and research what I don't. You're just being lazy at this point. You want to complain and whine about not having other settings, but then moan and winge about the settings we do have being problamatic, claiming that you are trying to be the change you want to see...but you're not. You aren't being that change. Do the research, the reading, the notes, the hours and hours of pooling over books at libraries and such, then publish your own material or play it with your parties yourself. You can't have your cake and eat it too... The whole thing about Avatar, it was written at a time where terms like "Cultural Appropriation" and that weren't around, and even now I still hear people in some circles claiming it's an issue that two white dudes wrote it, despite the people complaining not understanding the amount of research they did. Also their water tribe, yeah...while there were people that were happy with the representation, there were those that were pissed off with it that were Inuit and Yukip, and they said they did nothing but profit off the very basic stereotypes and knowledge of their cultures...so that's not really a good argument for your statement here...
The "Medieval Europe" trope they have going on now in DnD for Faerun is just as riddled with stereotypes...you do realize this right. But somehow that's okay, somehow that doesn't upset you. it only upsets you because you have played it out. If you looked into actual medieval history, you would come to realize that the reason why each town has a castle or a ruling lord, is because when that ruling lord got permission from the king to build on that land, he was instructed to build a fortification that the king could use as defense, not just for grandeur. Well castles and that don't build themselves in a day, they take YEARS and man power to do it; no CATS around here to build it. So people travel out from surrounding towns to build this thing, and with that they had to build homes because sometimes these projects took not just years, but DECADES, and they aren't traveling home just to have to go right back. They built houses for the people, segregated farmlands for the entire site, built stables for the animals that both pulled wagons and livestock, had sheds built for the masons, wood carvers, metalworkers, blacksmiths, because you needed the tools repaired and such, the segregated off a section of the river for fishing and bathing, and even sectioned off an area for bathroom facilities which they rotated out once full. So in fact an entire town sprung up around this place, that's why the castle is usually in the center of town. Also, the motes, they aren't just there for defense, they were actually where the old quarry use to be. The lord would figure out where the best patch of stone was, and would build on top of it because of the stone, then mine around it and thus would end up with the defense of a moat. My point to this, it took me LITERALLY 20 minutes of research back in the day in order to figure that out, that's nothing. I didn't wait for TSR or WOTC to correct me on the medieval Europe stereotypes of pedants being dirty and lords being cruel or the center of the town just because they were rich; I went and did it myself.
What you don't seem to understand, and the rest that claim these things are "problematic" is that unfortunately, that's how the average person is introduced to cultures outside of their own. It's seen as "Exotic" in the sense that they have never witnessed the culture before, nor have they had experience with it. Exoticism is literally just an overfascination and overcuriosity of another culture outside of your own. There is nothing wrong with that and it honestly shouldn't be discouraged. If you want people to learn about a culture, and the only way they have ever been introduced is through what you consider problematic stereotype, why then discourage the exoticism they develop while trying to learn about it? Again, you can't have your cake and eat it too. Sadly, most companies understand that the only way people are going to recognize something as being from somewhere else, on average mind you, is if they use the stereotype and style the item that way. For example, If you put a portly white man in nothing but overalls and rainboots, with one strap undone and no shirt on, living in a small house out in the woods, immediately people are going to fill in the blanks by adding no teeth, hair all over, dirty clothes, and then claim he's a hillbilly from down south in Alabama. They'll even throw in the "married to his sister" bit too. Thus, the item is recognized as being form the south and therefore the item sells as "Southern" Another example is a tall, lithe black man, wearing a leather belt with furs and leathers hanging down the front and back, chest bear with a wrap around his head. He carries a spear and a shield and has a piercing through his nose made of bone. Most people are going to immediately recognize this as African and thus the item sells as "African". Companies know that these types of things are recognizable and therefore they use them because their audience doesn't want to actually take the time to learn what the culture actually is, or they are too lazy to. Another thing, stereotypes come around from things that are true to some degree. The hillbilly thing, there are people in the backwoods that DO inbreed and marry their sister in the south, but that's not the whole of the south now is it? There are tribes and places in Africa that DO make jewelry and piercings out of bone, but that's not the whole of Africa is it? Stereotypes are a grey area, to call them problematic can be problematic in and of itself because you could be disparaging one culture that actually does what the stereotype is based off of and offending them. That's a whole different topic though for a different thread at a different time.
Point is, having a couple stereotypes as a way to get the average person to identify where they are or what the setting is going to be is fine, so long as the rest of it is actual knowledge and correct information...and you are willing to contend with the people that claim everything is wrong just because you aren't of that culture. The issue comes with people saying the entire thing is problematic and therefore shouldn't be done. Do you really think a company is going to want to sit there and do the research and investigation all for people to come back and still claim they are a problem and go on the war path? No, not really. They would rather stick to what they know and not bother with the PR headache and nightmare that it seems to be now. If you aren't part of that culture, no matter how much research and investigation you do, people are still going to tell you it's problematic or cultural appropriation or some stupidness like that, and try to get you canceled or whatever. No, they aren't going to want to deal with that and instead they stick to their own, that way they don't have to deal with people like that and the nonsense they throw out. I have had it happen to me, despite putting an ungodly amount of time into research and investigation on Japanese culture. It's one of the reasons I stopped running games for anyone outside of my family now.
Do it yourself, if you really want to be the change you want to see, do your own research and make your own worlds based on these cultures, because WOTC isn't going to do it, no one else if going to do it for you. No one wants to put that much work into it only to have some group with some nonsense about them being some form of -ist, -ted, while expressing some form of -ism, all while the same people shouting about it try to tear them down to ashes over it. It's literally not worth it.