He is using his mount to attack, thus electing to duel as two on one combat. Hardly seems worthy of a Paladin that agreed to an honorable match.
The mount is part of the Paladin's abilities. If you were dueling a wizard that specialized in summoning would you expect him to not use him summoning spells because you would be outnumbered? It sounds to me that you problem is that you're outmatched .
Showing up for a duel mounted seems kinda unfair, to say the least.
He didn't show up for the duel mounted. He showed up for the duel, then took an action to call his mount.
If said Paladin were fighting an evil opponent and cast the spell Holy Sword would you complain that its unfair because he brought a holy sword?
Plus, he is wanting the pillar to not affect his charge rout, agreeing that it provides cover, but that he can still charge. Seriously not seeing his logic there at all.
You're taking cover behind the pillar... but how much? If you're taking complete cover than your line of sight is blocked and you can't see him. If you're sticking halfway out looking at him I could see his reasoning. In real life, if the two of you could see each other (your standing at the edge of the column), he could probably charge you. He would be able to charge to any adjacent square and hit you, just as if he moved next to you he could still hit you. However, D&D is now a game of minis, so its not longer true. In order to charge he has to move to the nearest square from which he can attack you, which is occupied by the column. So the answer is that he can't charge you. BTW he also has cover against you from the column, so he also gains +4 to his AC against your ranged attacks.