The thing is, people say they want magic items to be rare and special. They say they want their players to be impressed and to treasure their rare, valuable +1 Longsword. But the problem is, no matter how rare you make them, it's still just a +1 Longsword. +1 just isn't that much. A fighter who rolled better than you could have a +1 to hit and damage just from his higher strength, and that's at first level! A +1 weapon is crap. It's practically worthless. No matter how rare it is. Great, I have a 5% greater chance to hit an enemy. Like I'll ever really even notice.
If you want magic to be rare and powerful, the way to do it isn't by making low-powered items rare and difficult to find. It's by eliminating the low-powered items altogether. If ALL the magic weapons in the world are +5, than all the magic weapons in the world will be rare, insanely powerful, and priceless. Adventurers would be lucky to ever find one.