There is no such thing as hands-free casting if there is a somatic component. If the spell includes both M and S, you must use at least one hand to manipulate either the true material component(s) or a focus. Simply wearing it is not enough when an S is specified.Considering that the form an "arcane focus" takes is deliberately left vague, and that a Warlock can freely choose the form of their pact blade, I think I'd allow it. A Wizard could use a crystal pendant as a focus, keeping both hands free except maybe momentarily when casting, so I don't think a Warlock using a sword as a focus is out of line.