You'd be right. Its a touch attack, you touch the armor but don't do damage. I think discussions on touch spells with unarmed attacks (ie a monk with Shocking Grasp cast), have decided that you could do it, but you use full AC and the touch misses if the attack misses; though I could be wrong on...