Some other posters touched on something I was going to say: The etymology may point to something older and less exciting than what we are looking for.
Simply put, the word 'wizard' derives from the Middle English wise + ard, meaning a kind of philosoher, or sage. Such 'wizards' were invariably found near stores of knowledge (books), which were usually housed in monastaries or other castle-like structures. These storehouses often took the shape of towers, and since philosophers are given to study (and not the most sociable in society), the notion of the isolated scholar secluded in his tower is not an unreasonable image.
Moreover, the tower as metaphor, something already touched on in this thread, is a symbol rich in meaning. A separated structure, in this case a solitary tower, is really no different than the scholar himself. A wizard is a person whose knowledge isolates him from the members of the community. A 'wizard's' knowledge is, symbolically, something powerful, imposing, and inaccesible to all but the most skilled and determined in society, and because of that, the wizard stands alone. And as someone mentioned before, the practice of alchemy and astronomy must have seemed, to the uneducated mind, akin to magic.
So the whole notion of the wizard and the tower may be older than we think, stretching as far back as the beginning of written language and the early structures where the 'wizard' isolated himself for his studies. At least, that's my take on it.