Just to point out the bit about material components in the section on Casting a Spell:
(underline mine)
RAW here is that a caster can use a component pouch "in place of" specified components, except for those with a gp cost (and arguably, those that are specifically noted as being consumed by the casting).
It also says [PHB pg 205] that "If you can't provide one or more of a spell's components, you are unable to cast the spell". The logical conclusion is that you are using a material component that you have taken from your component pouch.
This clearly will not work if the pouch is empty and the RAW says nothing to establish that the component pouch has any components in it. Therefore, if your DM goes by literal RAW, you may have to fill that component pouch first. Admittedly this makes a component pouch more of a "trap option" than 5th ed usually goes for, but it sure seems like the way Gary Gygax would have intended.