Wolfspider said:
Yes. You yourself point out the two situations where bucklers can be used with weapons: "while using an off-hand weapon" (two weapon fighting) or "while using your off hand to help wield a two-handed weapon" (two handed weapon fighting). The feat applies to attacking "with an off-hand weapon," the same term that is used in the buckler description and the feat description.
Actually, not quite. The rules text (ie. not flavour text) in the 'Benefit:' section of Improved Buckler Defense says "When you attack with a weapon in your off hand, you may still apply your buckler's shield bonus to your Armour Class."
In comparison, the buckler description states "You can also
use your shield arm to wield a weapon (whether you are using an off-hand weapon or using your off hand to help wield a two-handed weapon), but you take a -1 penalty on attack rolls while doing so. This penalty stacks with those that may apply for fighting with your off hand and for fighting with two weapons.
In any case, if you use a weapon in your off hand, you don’t get the buckler’s AC bonus for the rest of the round."
Now, I can understand interpreting it the way you do, however to me it looks like the buckler description is implying that using a two handed weapon qualifies as "a weapon in the off hand", which meets the criteria for the rules text portion of Improved Buckler Defense.
To make sure I wasn't just biased because it looked like a pretty nice feat, I showed the 2 passages to my DM and asked for his decision without making any comments on the matter to influence him. He also thought it should work with 2 handed weapons.
(In the end I decided not to take it tho; I ran out of feats :\ )