To answer this RAW, we need a specific quote that defines "end your turn" and not simply that explains the end of your turn things that happen. With such a quote, I concede (RAW), but without one I would postulate that "end your turn" implies you need to take actions during your turn.
So you're saying that your turn can end without you ending it?
I would have thought that "You end your turn adjacent" and "You are adjacent at the end of your turn" were pretty synonymous, myself.
And we know from the PHB that "The end of your turn always takes place, even if you’re unconscious".
Another example, does Readying end your turn? I think that the mark would disappear (immediately) if the paladin, say, shifted away and readied to attack him if he came closer, rather than resuming when the paladin was able to attack later in the round.
As far as I can tell, Readying is just a standard action. It looks like you could Ready an action, then move, then take a minor action, then end your turn... and if at some point the trigger is satisfied, you can then take the readied action as an immediate reaction.
I can't think immediately of any reason you'd want to, especially since if the trigger occurs during your turn, you can't act on it... so normally, you'd Ready at the end of your turn.
Regardless, if the Paladin readies an attack and does not end his turn adjacent, he has not fulfilled the requirement of the Challenge even if he attacks later in the round with the Readied action.
Similarly, if the Paladin has Challenged an enemy, and on his turn Delays, he'll fail to meet the Challenge requirement... because when you Delay, end-of-turn-bad-stuff happens immediately.
-Hyp.