SRD - Invisibility said:
The spell ends if the subject attacks any creature. For purposes of this spell, an attack includes any spell targeting a foe or whose area or effect includes a foe. ... Causing harm indirectly is not an attack. Thus, an invisible being can open doors, talk, eat, climb stairs, summon monsters and have them attack, cut the ropes holding a rope bridge while enemies are on the bridge, remotely trigger traps, open a portcullis to release attack dogs, and so forth.
A summoning spell doesn't target your foe, it brings around an extra creature. While the lasting
effects of Summon Monster I can wander off at your command and attack willy-nilly, the spell itself doesn't actually target your foes. Spiritual Weapon doesn't actually
target your foes either, technically... it just summons up a force sword/hammer/etc to strike willy-nilly. As in Hypersmurf's example, Flaming Sphere doesn't target your foes either, it simply creates a flaming ball you can roll around. But there's one fundamental difference.
When a summoned creature attacks, you have nothing to do with it. Sure, you just cast the spell to bring it to this useful spot on the Material Plane, but you don't directly control each of its attacks. A summoned creature listens to you talk, sure, but that's it. Telling a summoned creature to eat dirt isn't "directing a spell effect," but rather simply good-old, free action "talking."
Spiritual Weapon and Flaming Sphere both create an effect, but it's by your direct targeting that the effect works at all. I think it's that distinction that breaks your Invisibility; talking to a summoned creature (or an ally, like mentioned earlier) may be hostile, but it isn't attacking. Directly targeting a foe with an effect you created does.
I'd allow you to keep your invisibility if someone jumped onto the flaming sphere, though.