Quoting out of order here...
As for magic items, I usually handle that by having the character that is exiting the party leave behind anything that is "owned by the party" and take whatever is "owned by the character" and have the new character arrive with items appropriate to the campaign's level and type on the helpful table in the DMG on page 38 or so.
This, exactly.
Switch with no XP penalty - the player should have a character they enjoy playing, not be incentivized to just quit the game and join up for the next campaign because they weren't digging their original character anymore. And it is a penalty to the entire party to be stuck with a lower-level character, so it isn't fair to everyone else to levy that penalty to someone who is trying to stick with the campaign and have fun.
But not this; not often anyway.
First off, we need more information about the party and campaign in which this change is occurring. Is there already a level spread within the party? (if yes, new character comes in at level matching the lowest established character) Has the same player done this sort of thing before? (if yes, and it's because the player gets bored of any character after a few sessions (I've met these types), consider giving an xp penalty to encourage some stick-with-it-ness) Has anyone else in that game ever cycled characters in and out like this, and if so what's the established precedent?
Second, we need a bit more information about the character coming in. Does it already have a played history in the campaign, i.e. is it a previously-retired character coming back into the party? If yes, it comes in either at whatever xp it was last seen at or at that plus whatever xp it has earned since (you'll need to set aside some time with the player to update the returning character, if only to see if it's survived) with whatever possessions it had when it left again plus whatever it might have acquired since.
This character cycling happens in my game all the time - usually between adventures - and you know what? I kinda like it. It helps keep things from getting stale as the party makeup is always slowly morphing, and as new (as opposed to returning) characters come in at level matching the party's lowest it helps (in an odd way) to slow the overall level advancement...which is also good when you're trying to make the campaign last as long as it can* before the levels get too high and the wheels fall off.
* - obviously, if you're running a tight adventure path that you want to get done in 6 months this is irrelevant; but I don't run games like that.
Lan-"I always have reserve characters rolled up and booting around somewhere in whatever game world I'm playing in, and I'll cycle them in and out of the party as chance permits"-efan