That's fair. I suspect that this ambiguity from Tolkien is also intentional; he emphasizes the supernatural aspects of Sauron: his ability to act at a great distance; dominate minds; control the snows of Caradhras (possibly); cause Orodruin to erupt; blanket the lands with darkness etc. In that sense, the Eye is also a cipher for Sauron's capacity to effect his power; to assert his will upon the world.
I think the reason that we never see a fully reified Sauron, and that this (physical form) is only ever alluded to, is precisely to convey this mysterious, tremendous power which is beyond the ability of mortals to grasp.