Well, you only lose the extra attack for one level, until you add either another cleric or fighter level, or something else that gives you +1 BAB. I thought in your original post you meant losing the one extra attack for good, not by deferring it for a level, which to me isn't a "loss", per se...