Del Rey actually has a good set of three books which comprise just about all of his stories. If you head to Barnes & Noble, I'm sure they'll have a few in stock.

I'd start with The Best of H.P. Lovecraft: Bloodcurdling Tales of Horror and the Macabre. It's got quite a few of what I'd consider his best stories: The Call of Cthulhu, The Haunter of the Dark, The Shadow over Innsmouth and so on.

The second one I picked up was The Dream Cycle of H.P. Lovecraft: Dreams of Terror and Death. It contains the majority of his "dream cycle" stories, including two of his longest works, The Dream Quest of Unknown Kadath and The Case of Charles Dexter Ward.

The third book has a lot of his less Cthulhu-inspired stories and poetry, but it also contains the third-longest work of his, At the Mountains of Madness.

Like I said, I'd start with Bloodcurdling Tales.... The Call of Cthulhu is possibly my favorite story ever. His use of grammar and vocabulary in that one is just incredible, and the whole story just has this uniquely eerie atmosphere you won't find anywhere else. I'd recommend just jumping right to it. Start off on the best foot possible.