Very much worth it. All four books give the essential expansions to the basic game. There is more in those books than you'll need, and some material that moves very much towards specialist territory, but I've never regretted getting them.
The books see constant use in my games.
Here's what I can remember being used of the material:
Complete Warrior
Classes: Hexblade, Samurai, Swashbuckler
Prestige Classes: Cavalier, Invisible Blade, Ronin, Spellsword
Feats: Extra Rage, Extra Stunning, Hold the Line, Improved Buckler Defense, Improved Familiar, Improved Rapid Shot, Improved Toughness, Divine Might, Divine Resistance.
Complete Divine
Classes: Favoured Soul, Spirit Shaman
Prestige Classes: Radiant Servant of Pelor, Seeker of the Misty Isle
Feats: Augment Healing, Empower Turning, Practiced Spellcaster, Rapid Spell, Cheetah's Speed
Complete Arcane
Classes: Warlock, Warmage
Prestige Classes: Mindbender
Feats: Extra Edge, Draconic Heritage, Sudden Empower, Sudden Maximize, Sudden Silent, Sudden Still, Sudden Widen
Complete Adventurer
Classes: Ninja, Scout
Prestige Classes: Animal Lord, Dungeon Delver, Thief Acrobat
Feats: Extra Music, Jack of All Trades.
(The players in my games tend towards the fighter classes - arcane and divine spellcasters are quite rare)
Cheers!