I am hardly the expert around here, but I have been DM'ing Greyhawk off and on for over 20 years, so I am going to put in my $0.02 here.
I think you should track down a copy of the Living Greyhawk Gazeteer. This will at least give you a good solid overview of the world. I think that it is the most important print source you could find. There are SO many other print sources that could be considered "essential" that you could easily go broke buying them online. The next thing I would do is read as much as possible over at
http://www.canonfire.com/ . There is a wealth of information available there. After that, head over to the WOTC website and visit the Greyhawk forums. There are some people who are regulars here whose advice and recommendations you should heed. In fact, I would search for Greyhawk information by Grodog (who also has a Greyhawk website), MerricB and Eric Noah. Those three are the ones whom I would consider the top 3 experts on the setting that visit here most regularly (I know I am leaving people out, but those three seem to post here more regularly than some of the other Greyhawk experts and a quite helpful and forthcoming with Greyhawk info). Also, while on the WOTC website, check out the Living Greyhawk page on the RPGA pages. Many of the regions have excellent, informative websites and there are many fansites out there as well.
I think these sources are a great start for anyone new to the setting. You could do worse than starting there.
I love the setting and am excited that it seems to be experiencing a renaissance of popularity recently. I still have hope that Eric and Paizo will continue to support the setting unofficially. Hopefully someday there will be a reprint/update of what has been published so far.
Good luck and happy hunting!
DM
