If you want to be truly vile, have the spider follow them around invisible and start killing people they talk to or encounter. Have the city watch come 'round after the first and say, "Hey, we're just interviewing everyone we can find that encountered the victim yesterday. Did you notice anything strange?" Light-hearted, no suspicion. With luck, your PCs will start to investigate this on their own, which will put them in suspicious places later on. Never have the spider attack a guardsman or simliar NPC, but small children they encounter, shop owners, barmaids, etc and make it someone the PCs converse with in public. Lots of witnesses. One every day or two. The guards will get suspicious since they'll constantly be coming to interview the PCs about the latest victims. Now, have the spider start phasing in at the edge of their vision. Ideally, only let one PC see it the first time. As this continues, the guards should become more and more convinced that the PCs are murdering townsfolk left and right, and the best story the PCs can come up with is:
"It wasn't us! It was the giant spider in leather armor! You can only see him when he wants you to see him."
"Sure Mr. Hero. Why don't you make this easy on yourself and come quietly."
Z