Well.. from a purely semantic standpoint (which most rules are)
Since you have a choice (note: "you can get one extra") and you decide to decline this choice, then you are not "Fighting this way". And would not be subject to the penalties to offhand attacks.If you wield a second weapon in your off hand, you can get one extra attack per round with that weapon. You suffer a –6 penalty with your regular attack or attacks with your primary hand and a –10 penalty to the attack with your off hand when you fight this way.