This is not correct actually...
Casting an offensive, non-damaging spell on someone (such as Charm Person for example) is also considered an attack. Also, if I strike at someone and miss completely, that too is considered an attack.
Otherwise, you would remain Invisible if you strike at...