You're absolutely right; in fact, his physical prowess might even be greater in the Tower of the Elephant, he is, after all, a young wastelander at the time and not a king, but his experience, his "level" is much lower. He has, and I quote "really good stats and attitude".