By harm do you mean the use of force against others?
Otherwise, that line of thinking seems to break down in my mind when we overly simplify harm as a simple binary when it's a complex continuum including persuasion, coercion, inducement/threat of the use of force, and the actual use of force...