Come to think of it, there's no reason I can think of why an opponent couldn't use the flaming sphere as a source for a pyrotechnics spell.
Odds are the baddies will be moving away from the sphere (looking the other way) while the sphere's caster and his party members will be looking towards it. An enemy spellcaster in the rear can use a pyrotechnics to blind (most of) the party. Guess what way the baddies will be running then.
That's it, make it an enemy sorcerer. As long as your wizard is directing the sphere the party's liable to be blinded. (BTW the sorcerer doesn't actually have to be in sight of the party, he can stay hidden as long as the sphere is in his sight).