Dude, if it's not a fool it's an uncharismatic dude failing a save against charisma. If it's not an uncharismatic dude it's a weak one rolling vs Strength.
If it's neither of these, it might be a clumsy fighter, and so forth.
The problem is that everyone fails 95% of the time against a wizard (or, really, anyone with a decent selection of save or suck effects ) that knows what his opponent's weakness is (and you're pratically bound to have one unless you spend a bunch of resources to make up for it. In which case your second-to-worst stat becomes your new weakness, and your chance of success increases from 1/20 to 1/10, congrats). And quite a bit of SoS are basically "I win" buttons.