Take into account that setting books are expected by WotC to be niche products, they don't sell as well as other books, but they still very profitable, and they don't cause burn out like releasing adventures too fast or lead to complaints of bloat, so doing a bunch of setting books allows you to increase the amount of books your selling without the consequences and risks of other higher selling books.
Also a lot of MtG setting books likely sell at MtG LGS instead of Amazon.
There are no "player options" books: literally every book with player options has multiple tines more DM information...which makes sense, since WotC has said that DMs do most of the buying, either individually or corporately for the table.
Again, live sales, but also consider: Magic is a very game store centric community (even though most people who play have never been to a game store), so the hardcore Maguc fans would buy where they get their cards...not Amazon, for the dedicated fans.
The 3e sales strategy was pretty much built around that, as was 4e - cranking out a new player option or monster book every month was the way you hit the target numbers and kept everything going (and since there were more players than DMs, more player option books than DM tools was also the way you hit those numbers).
But the thing is, they didn't hit their numbers: 3E, 3.5, 4E and Essentials all crashed and burned within a few years. The 5E product strategy was explicitly designed around trying to do the opposite of what had failed... and here we are 8 years later, with Rise of Tiamat still in print.
The 3E PHB was published August 10, 2000, while the 4E PHB was published December 5, 2008. That's 8 years and 4 months.
Hoard of the Dragon Queen was published August 19, 2014.
That means it will hit the combined publication lifetime of all of 3E at Christmas this year. Currently it's only the #6,096 selling book on Amazon...but it is still selling, and you can find it in stores. That's pretty shocking, all things considered.
Well, not out of malice or anything: that's the gobsmacking thing: absolutely no 3E book had as good a run as Hoard of the Dragon Queen, one of the lesst of the 5E books by most measures. Indeed, Tales of the Yawning Portal and every book from 5E that is older than that has outdone every 3x book, including Core!