It's not a double AOO at all, I'm not sure why you see it that way. As was mentioned, on his turn, could not the character simply attack him (instead of readying an action), and then get an AOO in the caster's turn? That's two attacks on the caster, why is readying an action to attack the caster...