The basic problem is that from around 1989 to 1998, TSR produced almost nothing worth owning. Those few items of quality that they produced were swamped under a ton of garbage with low production values, low density of information (big fonts, wide margins, spurious and repetive artwork, ink bling), limited utility, poor writing, bad editing, and no play testing.
It's notable that there is almost nothing that is 'classic' from that era.
Now that I strongly disagree with.
Dark Sun, Spelljammer, Planescape "weren't worth owning"!?
"The Illithiad", "The Sea Devils" you are trying to tell me were crap?!
Um, no way!!
Now, you're right that there were definately stinkers at times, but here's a point many folk don't get:
YOUR stinker maybe MY fave!
now, I'd say the last TSR boxed campaign setting for the Forgotten Realms was really poor quality, IMHO as an example of what you were speaking of, quality wise.
Compared ot the 1st "grey" boxed set, it was awful! lousy illustrations, poor printing/font and layout etc.
And I can well believe TSR didn't calculate production run amounts to what was economically viable (as said it wasn't well run by all accounts)
I foudn many items to be fun, useful or whatever.
"Fun" is important, wish folk would remember that at times like the collectible cards (handy for quick NPCs and generla interest value).
"useful", like the Encounter Cards, that was handy.
"The Illithiad" was outstanding. So they sure could make quality.
What angered me as a customer though were the damn metaplots in novel lines so they could re-write settings so they could sell newer boxed sets after screwing up perfectly damn good settings, GRRR!
"Too much stuff" well you weren't meant to buy it all, folks!! Didn't this occur to anyone else?
