The Gazetteer series from the late 1980s are the definitive books on the setting. You can find more information on them at RPG.net here. The first one - GAZ1: The Grand Duchy of Karameikos - is widely held to be the best of the series and sets the benchmark. Lots of useful information in these and I reference them regularly.What books are ones to look for, Mystara-wise?
I personally did not really discover Mystara until the mid-1990s however, when TSR brought out this box set. It is, in essence, the same information as GAZ1 but repacakged for the 90s and for 2e AD&D. I don't think it was a big seller but I can distinctly remember buying it from a small gaming shop in Scarborough!
Now, my favourite Mystara/Known World products was undoubtedly the Poor Wizard's Almanac series. Each volume in these series covers an entire year in the campaign world and is a true almanac. In these, you will find detailed information on the nations of the world, potted biographies of major figures, information on trade and economics, details of the movements of the world's armies and (my favourite part) details of events on every scale (local, regional and global), many that arc over several months or even years and most with ideas on how to involve your players. These books contain a wealth, several wealths, of game ideas and are a rich source of game world info. If you only get one book, get one of these.
You can get a taste of these last by looking at the community-authored versions for subsequent years. These carry the product line forward and are just as useful and interesting. You can find them here. This website - Vaults of Pandius - the Official Mystara Homepage - is another excellent resource for Mystara material. You can find everything here, from the demographics of turn-of-the-millennium Thyatis to the ethnography of all the languages of the Known World and everything in between. Well worth a browse.