Presumably because the older books are out of print, and no longer available in most shops - out of sight, out of mind - so providing a concise setting book for Khorvaire (aka Eberron Adventurer's Guide) would be beneficial for new or returning players who would potentially want to play Eberron in 5e.
I would like to see books similar to SCAG providing a sort of lite gazette for the Worlds of D&D. That may even be one way to approach a setting book: Worlds of D&D. A single hardbound book providing some info and basic mechanics on the various famous settings of D&D that aren't FR.
Oh absolutely... if WotC was to release any setting material for the non-FR settings, I definitely think it will be in either a large tome that has significant chapters on the "big five" or so settings with both setting info and mechanics... or they decide to do smaller SCAGish books for the settings individually.
The one thing I
don't think will happen though is the full and complete "Campaign Setting" tome that many people keep wanting/demanding for even Forgotten Realms. Where (like in the 3E FRCG) they have write-ups on every kingdom in the Realms, there are huge sections on Realms history, on magic, and religion, and the gods, etc. etc. If they aren't willing to spend their production budget of time and money to make a tome like that for the Realms (a setting you could
possibly justify because indeed the timeline advances so there is conceivably "new stuff" going on in a lot of those areas)... then they are NEVER going to do those giant books for Dark Sun and Eberron because they just did them for 4E, and
nothing will have changed (other than game mechanics for PCs and the statblocks for NPCs).
At that point... easiest and cheapest thing to do (for both WotC *and* the players would would like to play in those settings) is for just the "Worlds of D&D" type of book to point to DMs Guild and tell people "If you want even more detail... the setting info you can get from any of the setting books released during 1E/2E/3E/4E are all still valid, unchanged, and because they are PDFs of older books, much less expensive for you the consumer."