At my table "switching weapons" is a single object interaction and I think that was even clarified somewhere as being the intent of the rules (no, I don't have a reference).
We simply decided that dropping a weapon should still be a free action, so that a player could drop a weapon and draw a new one in a round. Then as a DM I just decided that it seemed like pointless nit-picking and somewhat immersion breaking to have all these heroes with magic weapons littering the ground with ancient artifacts during fights.
I believe this is still within the spirit of the rules, they're only doing one thing "swapping", but they can't open a door and draw a weapon etc.
I have also been easy on this, and allowed characters to swap weapons without using any action of its own. It does feel within the spirit of the rules to me, just like the movement rules in combat are now a lot more free than how they were in 3e.
I don't even think the RAW is pristine clear in this case. It does seem to suggest that if you store away a weapon and draw a second one, you should be spending your action on this turn, which means losing all your regular attacks (but not your bonus action) for the current round. But then the RAW could also be interpreted in a very annoying way, and start arguing for example that you can draw
one arrow as part of the attack but not
two because technically it's 2 different objects, or that you can't draw your bow
and an arrow in the same round for the same reason. Or they could be argued the other way around by saying that the "ammunition" property allows
each arrow to be drawn as part of its
attack (instead of
action).
Actually in the 3e era I used to be more strict about this sort of things, but it made sense back there when the game was more rules-oriented, and the rules were more clear: IIRC drawing was a move-equivalent action (but with a special discount if you had BAB higher than 0), sheathing and picking up were move-equivalent actions, and dropping was a free action; these meant that there was a simple logic that created tactical choices. In 5e it's a bit more fuzzy, and to me it suggests that your gaming group should discuss+agree+settle on some rules for consistency, but only if you think it's worth doing so (I don't, not on this area of the game, I don't find it interesting enough at the moment to be worth spending time discussing it with the players).
You could also choose to handle the entire subject narratively. I know that some gaming groups enjoy immersing more in narrative descriptions than rules minutia, and for them it might be better for the DM to decide depending on
what weapon they are using (e.g. drawing/sheathing a greatsword not the same as with a dagger), or other circumstances.