I think he comes from a line of Sith Lords, by blood. Makes the most sense to me.
After all, we know that Jedi potential tends to run in families (this was true, even before that midichlorian junk). We also know the Jedi tend to test children for that.
So if he was just had strong Force potential on a civilized world (he made Supreme Chancellor - he probably isn't from too far from Coruscant) - the odds are much better that he would've ended up with the Jedi, not stumbling across a Sith Holocron. Plus, I think it would've been hard for him to hide his training from the Jedi council, if he didn't have a real mentor.
Nah. I'm guessing Papa Palpy was also Sith, and the one before him, and that Palpatine has some trouble siring kids of his own, or he wouldn't need to outsource.
As for Jorus C'Baoth...in the EU continuity, I'm pretty sure Palpatine had the original killed. He did have a Dark Side corrupted clone of the guy guarding something important, though.