Is there a litmus test you can outline? Is it more or less "Does this subclass have a story?" Is it about whether there's a reason for it to exist outside of strictly mechanical options?
To try to get some clarity on this, let me give you a couple of examples of custom subclasses in my game. I have a fighter subclass that is the devoted defender. It's basically a bodyguard. Its subclass abilities are focused on defending a designated charge. Would that qualify as a how or a why subclass?
Or how about the warrior of Chaos- another custom fighter subclass. It gets various Chaos traits as it levels up, and can trigger chaotic, random effects when it uses things like Action Surge or Second Wind. Is this a how or a why subclass?
I'm interested in the distinction you're drawing but am not sure exactly where or how you're drawing it. The swashbuckler, for instance- it seems like it's more of a "how" subclass than a "why" subclass to me.