I bought PHB1/DMG1/MM1 as a boxed set. I've also got Arcane Power and DMG2. Most importantly, though, I have a D&DI subscription. Now, I only buy books like DMG2 that contain a lot of text that isn't in the Compendium. Pretty much everything in the *Power supplements shows up in the Compendium, so I haven't bought any more of those, or PHB2.
DMG2 has a lot of useful information on skill challenge design and includes rules for monster themes and templates, revised moster leveling/deleveling rules, and plenty more -- I think the only DMG2 content that's in the Compendium is the new traps and monsters.
In the case of Arcane Power, I couldn't bear to wait until the monthly Compendium update to see the contents, so I bought it. I've considered MM2+, because only the monster stat blocks are in the Compendium, not the images, descriptions, knowledge checks, etc.; I haven't needed it so far, but it's on my list of possible future book purchases. I'm definitely interested in the Dark Sun campaign setting books. I considered buying one of the Eberron books for the Alchemy rules text, but I decided that buying a whole book for a couple of paragraphs of content was probably not worth it. The other stuff I wanted from Eberron -- Artificer and Warforged stuff -- was all in the Compendium and Dragon/Dungeon articles.