This seems to be a common line of thought. Since elves exist, and they can see in the dark, then centaurs can climb? Since dragons exist, centaurs can climb? Since there are devils, centaurs can climb?
Here is the difference imho.
We have examples of real life things that can see in the dark. It is not hard to suspend disbelief that a species not human can see in the dark. We have examples of things that can live a long time. We have, through antiquity, been told about devils and dragons. There are clear rules regarding these creatures, and their presence/culture/history is explained ad nauseum.
A centaur climbing a sheer cliff wall or a tall rope ladder to the crows nest defies physics. If you want to give the centaur the innate ability to use a beefy version of Tenser's floating disc for its hind hooves. Cool. Or maybe an innate fly spell that just holds their weight, but doesn't allow them to fly. Then at least there is common sense backing the decisions up. I give all my flying dragons a distorted version of the fly spell. But to say there is a centaur Alex Honnold does break reality - because you are denying physics.
If you deny physics then all characters should be able to jump 1 mile. Why not? Let all characters jump from tree top to tree top without any ability or training. Or, better yet, have them be able to pick up castles. But of course, we generally don't allow that because we want our worlds grounded in physics.