Correct. Until we make contact with another sapient alien species, we can only guess as to the nature of intelligence from our own perspective. However, we did try that before to explain the difference in race in the real world a century ago and it didn't work out well. Even used in its most neutral and objective form, it argued that inherent limitations of certain races made them ill suited for certain types of tasks, showed they lacked the cognitive ability to understand right from wrong and made social integration difficult if not impossible.
In short, they compared them to animals.
And unfortunately, even if it was done innocently or for well intentioned reasons, that language is what is used when we compare races in D&D. An orc's biology and culture defined him as intellectually inferior, a social outcast, predisposed to chaos and evil, physically strong, living a primitive lifestyle and best suited for physical occupations rather than intellectual or leadership ones.
That language has been used before to demonize countless groups of people. It is a relic of bad science used to bad effects. The sooner that that kind of language fades into obscurity, be it for humans or orcs, the better.