Much like the thread on missing players, my answer depends on the style of game.
1) If I were running an old school 1E game, they'd start at 1st level (probably). That's just the way it's done. Gary would be proud.
2) If I sense the player is trying to bring in a "superior" character, or are just too ADHD to actually stick with a character, they generally come in a half or full level behind their current character. The half level penalty isn't bad enough to really hurt other players who might lose a character or have a "good" reason for switching. It really, really turns off the player trying to one-up the other players, though. It's also a compounding penalty for anyone who likes to rotate through PCs to a level that would annoy the rest of the group.
3) If the character has kinda run its course, doesn't easily fit with the other PCs, or it otherwise makes good narrative sense for the character to go, there's zero penalty. The old character takes his stuff and the new one comes in. Ditto for builds that have proven themselves to be broken (upwards or downwards) to an un-fun extent.
Note that I've used option #3, exclusively, for the past decade+. The longer I play, the more I gravitate towards groups that are friends, first, and don't jack with each other. We also get more resistant to anything that would break continuity. Option #2 was the norm when I was in college, because we generally had more random folks rotating through. I'd still be up for #1, but it would be as part of a totally different game than what I'm playing, now.