Do your players respect the bonds/flaws/ideals on their sheets? If so, maybe the offer is a very generous one -- the fates are happy to restore your friends' memories with the fulfillment of a quest (taking the Cloak of invisibility from some level-appropriate challenge). But when they do, the players with memories restored have to re-write either their bond or their ideal with something in favour of the fates:
-- "The Fates have chosen me to be their herald"; "The Fates saved me when no one else could"; "Giving myself to the Fates is the best decision I ever made"; "The Fates love me like no one else ever will."
The players will insist this is who they always were.