What sort of swarm are you talking about?? Rats, bees, crawling claws??? What weopons are being used?? Standard weopons or spells with area of effects??
IMO if the characters are using weopons then there'll be a minus to hit but you should be able to strike at the things without touching a companion but if you are trying to help them out then yes you would hurt them.
The best bet against the insect swarms is things like wind wall or fire based spells such as burning hands.