On all opposed attack rolls the opponent always uses his highest attack bonus. This is true even if you try to sunder his weapon multiple times in one round (if your BAB allows multiple attacks). So even though your BAB will go down with each attack, the defenders will always be at the highest.