I'd say Xander is the most obvious choice of the three mentioned so far (Xander, Willow, Spike) because he has no counterpart on Angel.
Willow's role of expert on magic and arcane matters is played by Wesley, and her character of the cute, innocent, brainy girl is played by Fred. Granted, Willow is a far more accomplished spellcaster than Wesley, but there still isn't a clear niche for Willow to fill on the show.
And Spike has no reason to hang out with Angel's gang. His reasons for hanging out with Buffy's crew is already pretty weak. Some might say that adding Spike to the show might take away from Angel's uniqueness. Oh, so we have another vampire with a soul on the team now?
The closest counterpart to Xander was Doyle, who was replaced by Wesley long ago. Doyle was the "everyman" of the show, despite his demonic heritage. The closest thing we have to that on the show is Gunn, but Gunn and Xander have little in common. So Xander could be added to the show without taking away from any of the current cast.