I agree that it's not an attack, but it IS "target(ing) a creature with harmful... spells".
You can fireball an invisible creature because you can target an invisible creature in the area of a fireball.
I hate retconning, and we've done it twice! (I forgive us, because this being a "test", we are trying harder than I think it normally "matters" in a "real" game to get it "right").
... But it goes to show that games of this high level are complicated. I mean, look at how many mistakes we've made, and I would think that we count as "very good" at this game!