I don't. I ask, "What does your character sheet say you are wearing?", and if I can't find some definite evidence of gloves then they aren't wearing gloves. Simple as that.
Because your objection is really pathetic, because if I can't tell the player that their PC is not wearing gloves then I also can't tell the player that they are not wearing gloves, in which case we are reduced to a Schrodinger's fiction where the PC is or is not wearing gloves only after they find out whether or not it would be useful to be wearing gloves.
You won't be able to sustain that with evidence either from manuscripts or paintings. As a matter of actual evidence, the medieval didn't paint themselves ubiquitously wearing gloves. You are now well into the rules lawyering stage.
So now you are at the rules lawyering stage where you are arguing since the rules don't support you, the evidence on your character sheet doesn't support you, and the evidence of history doesn't support you, that the rules are wrong and you are appealing to "common sense". Only, if the consequences of wearing the gloves were negative and it was the DM arguing that his common sense told you that you were wearing glove, you'd be vigorously siting how the absence of gloves on your character sheet, the absence of gloves in the rules, and the fact the medieval painted peasants working in the fields without gloves to be evidence your character wasn't wearing gloves.
I'm not even going to count the logical fallacies in that, it's such an absurd argument.
Basically we just have a player here that is waiting for you to disclose metagame information before they decide what they are wearing. As has been previously discussed, there are processes of play you as a DM can use to get around a player doing that, but fundamentally the issue is that even if the DM hasn't been clever and this issue has taken them by surprised, at the point that they imagine the scene with the player not wearing gloves the player must have some positive evidence of the gloves before they can demand as drastic of an action as a retcon.
The <insert profanity of your choice here> they do. The player typically if they have a specific vision of what their character is doing expresses it. Much more typically, the player begins by offering a rules proposition like, "I check for traps" or "I search the room", with absolutely nothing like a concrete idea what that looks like because they are engaging with the rules and not with the fiction. And most of the time that's often sufficient to adjudicate the scene which is why DMs let it pass and why players get in the habit of not concretely imagining things like what they are holding in their hands (leading to the ubiquitous problem of all PCs having five arms), or what they are doing with their hands. It's really rare that you have any player who imagines things like, "Well, since I have a sword in one hand and a lantern in the other, I sheath my sword before walking over to the desk and opening the top left drawer, holding the lantern up so as to get good light." It's only when prompted by something that they start trying to reify the situation.
None of which has a thing to do with what is on your character sheet, nor am I going to listen to an argument that requires me to count the number of fictional portrayals of characters with or without gloves. Again, in this situation the character is wearing a suit of normal clothing that does not explicitly mention gloves. Therefore, the GM is perfectly valid in thinking that gloves are not present or that if gloves are present they are knitted or fingerless or otherwise not intended to provide physical protection. If you thought gloves were so important, you'd have written "kid skin gloves (worn)" or "heavy leather work gloves (worn)" on your character sheet. There is nothing gotcha about adjudicating the situation using the most concrete information that the DM has available. And a gotcha GM wouldn't design the trap this way. A gotcha GM would have a curse that went off if you touched the chest while wearing gloves and/or boots, because the chest was a sacred reliquary that required the clergy to open it unshod and with bare hands.