Hypersmurf
Moderatarrrrh...
It seems to me that just holding the weapon imposes no penalty. If it did, then the "and claim an AC bonus" would be unnecessary.
Oh, I agree. Just holding the weapon does not cause a problem. But using it requires that you declare it as part of your attack action, whether you swing it or not.
Assuming the Full Attack Action:
If you are holding a longsword in your right hand but don't attack with it, and attack with the Defending shortsword in your left hand, you can benefit from the Defending quality without incurring penalties.
You can attack with the longsword with no TWF penalties, but you can't benefit from the Defending quality of the shortsword.
If you attack with the longsword and also wish to benefit from the Defending quality of the shortsword, you take TWF penalties. Even if you have four iterative attacks and only swing your longsword once and the shortsword not at all, you are using both weapons in your full attack, and suffer the penalties accordingly.
To me, that suggests that if you swing your longsword once and your shortsword once, even if you have four iterative attacks, you suffer the penalties.
-Hyp.