As an alternative to the *poof* mode is to create an adventure when the new player joins of a reality shift (ala Yesterday's Enterprise). Something happens (which you could lead up to on the last game of the leaving player) that shifts reality. The new one has the new player and everyone thinks...