At our table, if you know the spell, that is to say, if you yourself are capable of casting that spell, then you can recognize the spell being cast. If you are a spellcaster, you can make an arcana check. If you are not a spellcaster, then no.
But I'm a hardass. I don't ask. It all seems a bit...