I think stealing a component pouch is really a bit of a dick move, as we all mostly ignore the component thing because it's a drag.
If you actually have your players buying bat guano from the market than it's different.
But really, how do you think the players would react if they go to cast a spell and you say "sorry, you passive perception wasn't high enough to spot your spell components being stolen"?
A component pouch contains all the components needed to cast a spell. You do need to buy the pouch, you don't need to buy the bat guano because it is already in there. If you have the pouch you have all of them.
My DM absolutely requires either a pouch or a focus and it is important for balance to keep track of due to the action economy and handiness. If you are holding a focus, that hand is occupied. If you have a pouch, your hand is not occupied, but drawing out the component does take your free action. You can't do that and then draw a dagger too, you can't stow a dagger and then draw out a component. While the components themselves are for flavor only, the rules on using them are an important aspect of class balance and should not be ignored IMO.
I don't think it is a dick move at all. It is a creative move which is clearly within the rules, would be rewarded if I was DM and was rewarded by my DM the only time I tried it. That kind of thing is what it is there for and what it is intended for and creativity should not be punished.
As far as how would I feel - I had the same character nerfed by not having components because they were taken and I spent two sessions scrounging in the dungeon until I got to civilization and could buy components again. It is part of the game the only spells I could cast without components was light and shield. I could not even cast the message cantrip.
I still had fun, and I went through a lot to scrounge. We killed an enemy and cut a piece of his armor off to cast mage armor. When we came across a kitchen I asked if there was any butter or pork fat so I could cast grease.
I tell you what, I felt a lot better about that then I did about my character walking over a trap after I searched and going to 0 hps when we were pretty much out of healing in the party.