Player A no longer wants to be a dwarf paladin, now he wants to be a human ranger (or whatever). Do you allow him to just switch with no XP penalty?
Do you start him with less XP?
What if that dwarf paladin had magic items? Does the human ranger get them?
Sure something like this has been said. There's a million ways to handle this, and ultimately, it's however you want to handle it (not what any one or other table does).
Myself, they take an XP hit (which might not even be that much of a hit if they're already the lowest level member of the party) that puts them either equal to or 1 level lower than the lowest level PC in the party, depending on the power/level spread of the party.
Same for character death. Replacement/new PCs come in at equal to or a level lower than the lowest level "still going" PC.
So, in some cases, this is next to nothing. If the whole party is 3rd level, then they probably will start at 2nd level since it is trivially easy to get up to 3rd. If there are two 7th level PCs, a 6th, and 3 5th levels PCs, the "new guy" is probably coming in at 4th...and likely will be caught up to rest of them in a session or two. It's really just a judgement call, depending and varying from situation to situation based on circumstances.
Obviously, they also will have to wait until there's a place in the narrative that makes sense for this new character to appear. If it's "downtime" or in between adventures and the party is just going about town or wandering down a road to the next city, easy enough to start right up with "a mysterious stranger approaches you..." But, for example, in the middle of dungeon or half way through combat, this is much more difficult..."rescued prisoner" is probably my most often used play here. But even that isn't always doable. So, the player might have to wait until the party is somewhere this new character can show up on the scene...and yes, I have given players a week off here or there, knowing that that won't happen in a particular upcoming session.