"Special" is not the same as "better," and differences in performance are not the same as differences in inherent worth. Michael Phelps has both the massive training and dedication to be an Olympic athlete (or at least he did, he may be retired now?), and the small but meaningful advantage of his genetics, which give a tiny, tiny edge over others....but when you are already at the peak, the top 0.01%, those small advantages become much more significant. That doesn't mean Mr. Phelps is a more valuable human being than you or me.
Olympic athletes collectively are a special kind of people. Even if they don't win a single medal, to be part of the Olympics is an honor and something that should, by definition, mean you don't match the generic statistics for human beings. Physically stronger, hardier, more flexible, etc. Recognizing that training and selection pressure combine to shift the population statistics for Olympic athletes compared to general humans is not some elitist garbage elevating them as inherently more noble; it is simply a recognition that they are choosing a life that makes them different in some ways, and that succeeding at that life necessarily filters out some people who make the attempt but don't measure up.