Actually, if you have to "interact with an object" to double-tap with the same hand crossbow, you'd have to do the same thing even if you're wielding, say, a sword and a hand crossbow.
You'd get one bonus attack per combat (assuming the crossbow starts loaded), but from then on it will only be...