I am somewhat amused by the use of "the term weapon is subjective because that's just a label" to support an argument based on a rather rules-lawyer approach to defining such labels in the first place.
See, here's the thing. Bag of holding: the magic is on the inside. The magic enchants the space contained in the bag. So if you want to try and damage a golem with the space inside a bag, sure. I'll let you try. Spoiler alert: you'll rhyme with 'try'.
Potions: the liquid is magical, not the bottle. Bottle is nonmagical and the contents need to be injested.
Headband of wisdom: the magic works by interacting with your mind. If you want to hit a golem with a magical mental interaction then go ahead. You won't get far at my table.
Boots of speed: similarly, their magic works by altering your ability to move through space, once you are wearing them. The magic works only when worn. So again, sayeth the golem, come at me bro with your Gift of Slippers, see how much I hurt.
Bracers of defence: their magic works by helping to block or deflect or cause to miss blows from an opponent. This magic works when worn. Throw them at your enemy: they'll swerve off course. Hit the golem with them while they're still being worn? It's defensive magic - it won't activate offensively.
Ultimately any magic item is infused with magical power in order for it to perform its function. A magical potion performs its function and its magic manifests when it is drunk. A bag of holding's magic permits you to magically store things.
These objects' magic is inherent to their purpose.
As magic items are crafted or created, the one who crafts them in the process of creation is doing so with intent - concentrating that magical power to ensure this object (whether weapon, luggage, clothing, whatever) will be able to do/better able to do X job when worn, imbibed, commanded, or wielded.
It is taken by an act of will from the weave of magic, contained, mastered, controlled, given purpose in its creation - rather than being a bit of wild magic which has floated free from the weave (which is the magical equivalent of throwing your clothes off and running round the mountainside going "wheeee!").
A +1 sword, aside from being purely a mechanical in-game label, is a sword whose purpose (being a weapon) is magically amplified - better able to hit/deal damage.
A Frostbrand is a weapon whose function has been amplified by being able to deal a different sort of damage. But its function is still to damage.
An attack with a magical weapon is a magical attack; an attack with a non-weapon magical item is an improvised attack. An improvised attack is not a magical attack.