The text of the spell I believe states that you retain your own INT, WIS and CHA. Thus, if you get any of these abilities drained, you would carry that penalty over into another host. The same would hold true of level drains.
STR, DEX, or CON draining attacks would stay on the host body, not...