using an item to bypass it means the item can be lost stolen or knocked around. Goggles or a visor can be taken off someone's head or broken in a fight. the item could be stolen or confiscated due to in game reasons. Its not like you are just letting them have it for free.
Having the option to take away an item is pretty meaningless unless I'm actually willing to do it on a regular basis. I wouldn't do that any more than I'd take away a wizards spell book, have rust monsters eat a fighter's armor in the middle of the night. I'm not going to do that because it would feel vindictive and I'm not that kind of DM.
If you want to play a drow for RP purposes without the benefits I have a solution. Want to play a drow because it gives you advantages over other races while ignoring the penalties? Nope.
Oh, and you do realize you're responding to something I wrote 3 years ago, right?