If I am not mistaken, you can make an offhand attack anytime you want with your fist, it is just such a poor use that it never really happens. I might be wrong, though I don't see it destroying balance, as it does not meet the requirements for most abuse-able abilities like a Sneak attack...