It doesn't take an action to draw a weapon, it is a free interaction that you can do as part of your movement and action, similar to many other interactions (PHB 190).
Excluding Heavy weapons goes a long way toward making this more feasible. I'm still not a fan of the concept, however. I would...