Besides that there's nothing saying that the remainder of the horse part isn't contributing to the climb.
When you're pulling your self up, there's plenty of work happening in your core, shoulders, back, etc. where horse muscles can be contributing. Also, assuming the human torso part can pivot forward, then the fore an hind legs could get involved in contributing to the climb as well, in similar ways that a humanoid climber's legs would be involved.
The difficulty then is range of motion and holding themself to the climbing surface, which would be difficult. Perhaps like difficult terrain.