I'd start with either of these:
The Call of Cthulhu and Other Weird Stories
http://www.amazon.com/exec/obidos/t...f=sr_1_1/002-4348101-6309663?v=glance&s=books
The Thing on the Doorstep and Other Weird Stories
http://www.amazon.com/exec/obidos/t...1/002-4348101-6309663?_encoding=UTF8&v=glance
Both are edited and have notes by S.T. Joshi, the most knowledgable Lovecraft scholar out there. His work on these editions is excellent, and the editions by Penguin have very nice selections of stories presented in the definitive corrected texts. The first contains some of his most classic short stories, while the second contains some great stories and
At the Mountains of Madness, which many consider his best work.
Whatever you do, don't buy the editions that DelRey has out there, they cost the same but don't have the corrected texts and the annotations found in the Penguin editions. The most popular of these is
The Best of H. P. Lovecraft: Bloodcurdling Tales of Horror and the Macabre which is what most booksellers will point you towards, since almost all the big stores carry it. Skip it and have them order one of the Penguin editions if they don't have them in stock, it'll be well worth the extra couple days you have to wait for it.