The OD&D boxed set and all of the supplements are available on pdf from RPGNow. The boxed set alone is all you
really need, but you'll probably want
Chainmail and
Supplement I: Greyhawk as well -- the former for the complete combat rules (even if you decide not to use them), the latter for many "familiar" rules, classes, spells, monsters, and items that are missing from the boxed set (even if you decide not to use them). The rules will tell you that you also need a copy of Avalon Hill's
Outdoor Survival game, but they're lying.
I'd consider "Best of The Dragon vol. I" (collecting articles from The Strategic Review and issues 1-14 of The Dragon) almost a must-have; I use it more than any of the later D&D Supplements. I'd also strongly recommend Judges Guild's
Ready Ref Sheets (available on pdf) and
First Fantasy Campaign (not available on pdf AFAIK) and a copy of the Holmes-edit
D&D Basic Rulebook (available on pdf), which isn't technically OD&D, but is mostly compatible, shares the same spirit, and is much easier to understand (and use to introduce new players) than the boxed set.
B2 was written for the aforementioned Holmes edition rather than OD&D-proper, but is about 95% compatible with OD&D (assuming you're using Supplement I) so there shouldn't be much/any problem using them together. The same goes for B1. Other than those two, you're going to have to look to third-party publishers for adventures -- the best coming from Judges Guild (Tegel Manor, Caverns of Thracia, etc.).
For active fan-discussion of OD&D, check out
The Knights & Knaves Alehouse and
Original D&D Discussion, and also be sure to check out the informative and thought-provoking OD&D blogs by
Delta and
Philotomy Jurament.