The problem came where I wanted to see if I could borrow an item to cast a spell with and give it back later. DMs ruling was that I wouldn't be able to do that as the costed item would be used up by the spell, but I don't think that should be the case.
Just to bring up another point about this particular part of what you said here. While I also agree that a component isn't consumed upon casting unless the spell says so... I do in fact agree with the DM that in some cases I wouldn't allow a caster to borrow a costed material component from someone else and then give it back. Not because it would be consumed... but merely because my belief that if you have a spell in your spellbook and have been working and training to cast this spell (in the fiction of the story), you would have had to possess the costed component at some point (and thus, why do you not have in on your person now?)
In other words... for my particular way of DMing... casting magic doesn't "just happen". You have to practice and learn how to do it. So if you are a 1st level wizard, you need to buy yourself that costed component with your starting cash because you will have needed that component the entire time you were learning how to become a 1st level wizard-- you can't just skip buying the component with the expectation that you'll borrow it from someone else and then instantly be able to cast the spell. By the same token... if a caster finds a scroll for a new spell that requires a costed component, I would require them to acquire said costed component for themselves, rather than just think they can borrow components willy-nilly when they need them.
It's a very fine line... but for me in particular, I feel magic is something you need to practice in and around adventuring (just like swordfighting is something you need to practice in and around adventuring.) And if you don't have the spellcasting materials necessary to practice (because you're just too cheap to acquire your own), then I'm usually not going to let you borrow them. Because within the story itself, I'd see someone asking to borrow your components would be like someone asking to borrow your toothbrush. It's like "Ew! No, get your own you cheap bastard."
(But in a final point... if it was an
emergency and a one-time thing-- big battle is coming up, old wizard is too sickly to fight, young wizard PC is going instead but had his material components stolen, old wizard says "here, take mine!"... that I'd allow because it would make sense within the story.)