I voted that I use only core stuff, but that's untrue because I do use supplements for my PCs. Only that the other option in the poll was more untrue

because I can definitely play in a core-only campaign without complaining.
IMHO the best in supplements is the feats, because even with the many core ones (how many are they, 100?), there are characters who may want to improve or expand their own abilities without resorting to combat feats, too generic feats, or stepping on the toes of other classes with metamagic (sorcerer's territory) or item creation (wizard's territory). Furthermore, the core feat chains are too much one-way, so much that basically all the strong melee characters have to go the Power Attack-Cleave route for example, without combat alternatives. New feats are almost always welcome.
OTOH Prestige Classes are cool but most of the time they fail to grant something really new, and instead they are a re-hash of features which are already available from core classes. Taking a PrCl only has the aim of getting those ability earlier or in a different order/combination without facing multiclassing penalties, or otherwise just because "you get more stuff" overall.
Furthermore, there's absolutely no need to take a PrCl for RP reason: it may be appropriate, but no one says you can't be part of the Red Wizards without entering the PrCl... it is actually the other way around!
The PrCl I really like are those which actually give you something unique and otherwise unavailable, and something which is not just a huge + on a common ability. The DMG's Shadowdancer is IMHO the perfect example of how a PrCl should be designed! It's not even more powerful than a core class...