Send the cat to hell, thus annoying Asmodeus who then sends out assasins against the party. Of course you could just keep having the cat come back and bother the party. Makes for a good song!
Reminds me of a disguised copper dragon that followed my party a long time... he played a kinda doglike lizard. Immortal: Well, he's not but he's not easy to kill either.