I would agree with most of the others who state the only 'must' have is the core book. I would throw in (like others) that the bestiary might make the list as well, but only for the GM.
Moving on to 'really nice to have' and I would throw in Bestiary 2 (GM only), Advanced Player's Guide, Ultimate Magic, and Ultimate Combat.
The rest of the books become a lot more situational. If you are a GM I would recommend getting Adventure Paths prior to even the Nice to Haves. Additionally if you are running something in an area covered in detail by another book, I would get that book. For example, I am running the Carrion Crown Adventure Path which is set in Ustalav, which is the subject of the Rule of Fear book. The Rule of Fear book is great for running a game in Ustalav, but don't pick it up expecting something like what Heroes of Horror was in 3.5, it is decidedly a setting book.
The companion line, which I like, is definitely more for the hardcore fan. If you are playing an elf and want to make them unique and come to life, then Elves of Golarion is great, but in no way necessary. Or if you are a completionist (like me), then you are looking at all these smaller books to fill out your collection.
Sean Mahoney