Yeah, it's originally from Hindu religion, the word अवतार (avatāra), meaning "to cross down" or "to descend", referring to a deity that has taken on an Earthly form.
As I said, D&D copied pretty liberally from pop culture, literature, religion, and folklore. The concept of avatars came from...