When the red flag flies, the locals know to go indoors, and let the mysterious ships come and go unobserved...
While they may indeed be smugglers, it's more a case of not letting the local folk know that flying ships from other worlds deliver the exotic cargoes that keeps them so well off.
Somethings it's better not to know about...in case you look the proverbial gift horse in the mouth.
