To look at it from a "big picture" perspective, I think that a handful of broken items / abilities is a necessary evil of a game designed for constant updates. People often talk about things like "power creep" and broken abilities, without talking about completely expanded options, new classes, new fun items etc. that come from game expansions.
Basically, if you're ok with taking some bad with the rest of the good then I think you're in for a rewarding game experience. It's only a problem when a subsystem of the rules is so broken that it hurts the whole game. For example you as a GM could remove a broken item like Bloodclaw weapons with little impact on the game, but you couldn't fix grapple from an earlier edition without some serious work (houseruling) on your part (arguably, the "expertise feat" issue is 4E's biggest bugbear in this regard).
The alternative would be to design your game in a way that overbalances everything to the point where you see little difference between individual classes, options, or items to the point where the sameness becomes bland and boring. It can be a very tough balancing act between keeping things in line with the maths, and also having creative freedom to introduce new options.
I accept this necessary evil and so do my players. We often have frank discussions about things like imba items / abilities and as a group we agree what's best for the game, by usually agreeing to remove the offending thing. It's never a one-man decision, but it's worked pretty well for us. It's probably the best advice on the matter I could offer.