By the time a party is rich enough to be buying those kind of magic items, or have a wizard casting Item spells, they'll have access to all kinds of other magic items, spells, and such that gunpowder seems like a very inefficient use of resources. Wasting item spells, Tenser's Floating Disc, that could be used on attack spells is just taking time and effort to use gunpowder like it was a fireball spell.Bags of Holding, Item spells, Portable Holes. It's not bulky enough to stop them from using it, at all. So, yes, they CAN chuck full powderkegs at an enemy. Telekinesis helps. So does Floating Disks, if they are clever. Heck, tie it to a Summoned creature and send it in.
Gunpowder does blast damage, not just fire damage. It will kill people with concussion, not just burn them.
We're not worried about knocking down buildings so much as doing a lot of damage to things... and a barrel of gunpowder going off in a red dragon's mouth is NOT going to be something it can shrug off easily.
Players can and will abuse gunpowder. Magic gives them the tools to do so.
Even worse... if fire resistance/immunity works against it, that means any character with Protection against Fire can walk into the middle of enemies not so immune and light off barrel after barrel with impunity, right?
You're presenting an idealized scenario that ignores the various tactical circumstances where that just wouldn't work.